[==========] Running 4 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 4 tests from WebserverCrawlITest
[ RUN ] WebserverCrawlITest.TestAllWebPages/0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20251027 02:47:23.641609 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
2025-10-27T02:47:23Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-10-27T02:47:23Z Disabled control of system clock
I20251027 02:47:23.667165 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:47:23.667618 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:47:23.667917 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:47:23.670686 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.254:35183
--webserver_interface=127.152.231.254
--webserver_port=0
--builtin_ntp_servers=127.152.231.212:43561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.152.231.254:35183,127.152.231.253:40515,127.152.231.252:43789
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www with env {}
W20251027 02:47:23.969159 1729485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:47:23.969666 1729485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:47:23.970041 1729485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:47:23.998049 1729485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251027 02:47:23.998337 1729485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:47:23.998525 1729485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251027 02:47:23.998703 1729485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251027 02:47:24.030987 1729485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:43561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.152.231.254:35183,127.152.231.253:40515,127.152.231.252:43789
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.254:35183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-0/data/info.pb
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:47:24.032430 1729485 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:47:24.033914 1729485 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:47:24.046594 1729491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:47:24.047694 1729492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:47:25.107551 1729494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:47:25.108675 1729493 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1057 milliseconds
I20251027 02:47:25.108836 1729485 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:47:25.110543 1729485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:47:25.114379 1729485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:47:25.115780 1729485 hybrid_clock.cc:648] HybridClock initialized: now 1761533245115729 us; error 36 us; skew 500 ppm
I20251027 02:47:25.116643 1729485 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:47:25.123392 1729485 webserver.cc:492] Webserver started at http://127.152.231.254:37631/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:47:25.124414 1729485 fs_manager.cc:362] Metadata directory not provided
I20251027 02:47:25.124625 1729485 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:47:25.125178 1729485 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:47:25.132568 1729485 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-0/data/instance:
uuid: "c4182f12cf5b4085b5f13f2a99304d3e"
format_stamp: "Formatted at 2025-10-27 02:47:25 on 5fd53c4cbb9d"
I20251027 02:47:25.134011 1729485 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-0/wal/instance:
uuid: "c4182f12cf5b4085b5f13f2a99304d3e"
format_stamp: "Formatted at 2025-10-27 02:47:25 on 5fd53c4cbb9d"
I20251027 02:47:25.143234 1729485 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.005s sys 0.004s
I20251027 02:47:25.149745 1729511 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:47:25.150882 1729485 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20251027 02:47:25.151298 1729485 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-0/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-0/wal
uuid: "c4182f12cf5b4085b5f13f2a99304d3e"
format_stamp: "Formatted at 2025-10-27 02:47:25 on 5fd53c4cbb9d"
I20251027 02:47:25.151695 1729485 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-0/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-0/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:47:25.219573 1729485 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:47:25.220795 1729485 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:47:25.221333 1729485 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:47:25.305557 1729485 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.254:35183
I20251027 02:47:25.305684 1729562 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.254:35183 every 8 connection(s)
I20251027 02:47:25.308115 1729485 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-0/data/info.pb
I20251027 02:47:25.314677 1729563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:25.317855 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1729485
I20251027 02:47:25.318265 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-0/wal/instance
I20251027 02:47:25.322602 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.253:40515
--webserver_interface=127.152.231.253
--webserver_port=0
--builtin_ntp_servers=127.152.231.212:43561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.152.231.254:35183,127.152.231.253:40515,127.152.231.252:43789
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www with env {}
I20251027 02:47:25.326162 1729563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:47:25.350395 1729563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } has no permanent_uuid. Determining permanent_uuid...
W20251027 02:47:25.352559 1729515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.152.231.253:40515: connect: Connection refused (error 111)
W20251027 02:47:25.358400 1729563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:40515: Network error: Client connection negotiation failed: client connection to 127.152.231.253:40515: connect: Connection refused (error 111)
I20251027 02:47:25.407895 1729563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } attempt: 1
W20251027 02:47:25.411188 1729563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:40515: Network error: Client connection negotiation failed: client connection to 127.152.231.253:40515: connect: Connection refused (error 111)
I20251027 02:47:25.479540 1729563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } attempt: 2
W20251027 02:47:25.482904 1729563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:40515: Network error: Client connection negotiation failed: client connection to 127.152.231.253:40515: connect: Connection refused (error 111)
I20251027 02:47:25.574258 1729563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } attempt: 3
W20251027 02:47:25.577358 1729563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:40515: Network error: Client connection negotiation failed: client connection to 127.152.231.253:40515: connect: Connection refused (error 111)
W20251027 02:47:25.629524 1729565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:47:25.630052 1729565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:47:25.630482 1729565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:47:25.682886 1729565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251027 02:47:25.683329 1729565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:47:25.683697 1729565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251027 02:47:25.684059 1729565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251027 02:47:25.720677 1729563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } attempt: 4
W20251027 02:47:25.726047 1729563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:40515: Network error: Client connection negotiation failed: client connection to 127.152.231.253:40515: connect: Connection refused (error 111)
I20251027 02:47:25.761279 1729565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:43561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.152.231.254:35183,127.152.231.253:40515,127.152.231.252:43789
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.253:40515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-1/data/info.pb
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:47:25.763572 1729565 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:47:25.765954 1729565 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:47:25.783233 1729806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:47:26.025477 1729563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } attempt: 5
W20251027 02:47:26.031026 1729563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:40515: Network error: Client connection negotiation failed: client connection to 127.152.231.253:40515: connect: Connection refused (error 111)
I20251027 02:47:26.578483 1729563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } attempt: 6
W20251027 02:47:26.610951 1729563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:40515: Network error: Client connection negotiation failed: client connection to 127.152.231.253:40515: connect: Connection refused (error 111)
W20251027 02:47:27.183879 1729805 debug-util.cc:398] Leaking SignalData structure 0x7b0800037520 after lost signal to thread 1729565
W20251027 02:47:27.387791 1729805 kernel_stack_watchdog.cc:198] Thread 1729565 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 397ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20251027 02:47:25.784106 1729807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:47:27.389251 1729565 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.605s user 0.660s sys 0.944s
W20251027 02:47:27.389948 1729565 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.606s user 0.660s sys 0.944s
W20251027 02:47:27.392503 1729809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:47:27.397603 1729808 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1606 milliseconds
I20251027 02:47:27.397708 1729565 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:47:27.399317 1729565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:47:27.403401 1729565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:47:27.404891 1729565 hybrid_clock.cc:648] HybridClock initialized: now 1761533247404803 us; error 80 us; skew 500 ppm
I20251027 02:47:27.406231 1729565 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:47:27.414393 1729565 webserver.cc:492] Webserver started at http://127.152.231.253:46759/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:47:27.415728 1729565 fs_manager.cc:362] Metadata directory not provided
I20251027 02:47:27.416028 1729565 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:47:27.416642 1729565 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:47:27.424254 1729565 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-1/data/instance:
uuid: "736beccd865d4700879d4dde904dfb56"
format_stamp: "Formatted at 2025-10-27 02:47:27 on 5fd53c4cbb9d"
I20251027 02:47:27.425940 1729565 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-1/wal/instance:
uuid: "736beccd865d4700879d4dde904dfb56"
format_stamp: "Formatted at 2025-10-27 02:47:27 on 5fd53c4cbb9d"
I20251027 02:47:27.437199 1729565 fs_manager.cc:696] Time spent creating directory manager: real 0.010s user 0.009s sys 0.002s
I20251027 02:47:27.445693 1729831 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:47:27.447299 1729565 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20251027 02:47:27.447718 1729565 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-1/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-1/wal
uuid: "736beccd865d4700879d4dde904dfb56"
format_stamp: "Formatted at 2025-10-27 02:47:27 on 5fd53c4cbb9d"
I20251027 02:47:27.448180 1729565 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-1/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-1/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:47:27.560596 1729565 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:47:27.562660 1729565 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:47:27.563395 1729565 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:47:27.671486 1729563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } attempt: 7
W20251027 02:47:27.680785 1729563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:40515: Network error: Client connection negotiation failed: client connection to 127.152.231.253:40515: connect: Connection refused (error 111)
I20251027 02:47:27.690055 1729565 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.253:40515
I20251027 02:47:27.690207 1729884 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.253:40515 every 8 connection(s)
I20251027 02:47:27.694000 1729565 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-1/data/info.pb
I20251027 02:47:27.694574 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1729565
I20251027 02:47:27.695004 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-1/wal/instance
I20251027 02:47:27.697857 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.252:43789
--webserver_interface=127.152.231.252
--webserver_port=0
--builtin_ntp_servers=127.152.231.212:43561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.152.231.254:35183,127.152.231.253:40515,127.152.231.252:43789
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www with env {}
I20251027 02:47:27.703475 1729885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:27.722376 1729885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:47:27.753247 1729885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:47:27.775688 1729885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } has no permanent_uuid. Determining permanent_uuid...
W20251027 02:47:27.778584 1729835 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.152.231.252:43789: connect: Connection refused (error 111)
W20251027 02:47:27.786561 1729885 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:43789: Network error: Client connection negotiation failed: client connection to 127.152.231.252:43789: connect: Connection refused (error 111)
I20251027 02:47:27.836207 1729885 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } attempt: 1
W20251027 02:47:27.840935 1729885 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:43789: Network error: Client connection negotiation failed: client connection to 127.152.231.252:43789: connect: Connection refused (error 111)
I20251027 02:47:27.909394 1729885 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } attempt: 2
W20251027 02:47:27.914108 1729885 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:43789: Network error: Client connection negotiation failed: client connection to 127.152.231.252:43789: connect: Connection refused (error 111)
I20251027 02:47:28.005569 1729885 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } attempt: 3
W20251027 02:47:28.010331 1729885 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:43789: Network error: Client connection negotiation failed: client connection to 127.152.231.252:43789: connect: Connection refused (error 111)
I20251027 02:47:28.153798 1729885 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } attempt: 4
W20251027 02:47:28.159417 1729885 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:43789: Network error: Client connection negotiation failed: client connection to 127.152.231.252:43789: connect: Connection refused (error 111)
W20251027 02:47:28.243207 1729887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:47:28.244068 1729887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:47:28.244899 1729887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:47:28.308179 1729887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251027 02:47:28.308637 1729887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:47:28.309038 1729887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251027 02:47:28.309429 1729887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251027 02:47:28.381428 1729887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:43561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.152.231.254:35183,127.152.231.253:40515,127.152.231.252:43789
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.252:43789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-2/data/info.pb
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:47:28.383503 1729887 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:47:28.386149 1729887 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:47:28.404289 1729897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:47:28.458859 1729885 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } attempt: 5
W20251027 02:47:28.463866 1729885 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:43789: Network error: Client connection negotiation failed: client connection to 127.152.231.252:43789: connect: Connection refused (error 111)
I20251027 02:47:29.011303 1729885 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } attempt: 6
W20251027 02:47:29.020779 1729885 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:43789: Network error: Client connection negotiation failed: client connection to 127.152.231.252:43789: connect: Connection refused (error 111)
I20251027 02:47:29.771277 1729563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } attempt: 8
I20251027 02:47:29.803633 1729563 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } has no permanent_uuid. Determining permanent_uuid...
W20251027 02:47:29.811015 1729563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:43789: Network error: Client connection negotiation failed: client connection to 127.152.231.252:43789: connect: Connection refused (error 111)
I20251027 02:47:29.876479 1729563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } attempt: 1
W20251027 02:47:29.881862 1729563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:43789: Network error: Client connection negotiation failed: client connection to 127.152.231.252:43789: connect: Connection refused (error 111)
I20251027 02:47:29.935339 1729563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } attempt: 2
W20251027 02:47:29.941613 1729563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:43789: Network error: Client connection negotiation failed: client connection to 127.152.231.252:43789: connect: Connection refused (error 111)
W20251027 02:47:29.804950 1729896 debug-util.cc:398] Leaking SignalData structure 0x7b0800037520 after lost signal to thread 1729887
W20251027 02:47:29.975656 1729896 kernel_stack_watchdog.cc:198] Thread 1729887 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 398ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20251027 02:47:28.405220 1729898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:47:29.976457 1729887 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.572s user 0.730s sys 0.807s
W20251027 02:47:29.976892 1729887 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.572s user 0.730s sys 0.807s
W20251027 02:47:29.980418 1729900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:47:29.981737 1729899 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1568 milliseconds
I20251027 02:47:29.981851 1729887 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:47:29.983527 1729887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:47:29.987758 1729887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:47:29.989212 1729887 hybrid_clock.cc:648] HybridClock initialized: now 1761533249989118 us; error 84 us; skew 500 ppm
I20251027 02:47:29.990448 1729887 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:47:29.998735 1729887 webserver.cc:492] Webserver started at http://127.152.231.252:38715/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:47:30.000150 1729887 fs_manager.cc:362] Metadata directory not provided
I20251027 02:47:30.000468 1729887 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:47:30.001168 1729887 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:47:30.009276 1729887 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-2/data/instance:
uuid: "947a379b7ea14817b0e501b4d845b6a3"
format_stamp: "Formatted at 2025-10-27 02:47:29 on 5fd53c4cbb9d"
I20251027 02:47:30.010970 1729887 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-2/wal/instance:
uuid: "947a379b7ea14817b0e501b4d845b6a3"
format_stamp: "Formatted at 2025-10-27 02:47:29 on 5fd53c4cbb9d"
I20251027 02:47:30.018054 1729563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } attempt: 3
I20251027 02:47:30.022743 1729887 fs_manager.cc:696] Time spent creating directory manager: real 0.011s user 0.011s sys 0.002s
W20251027 02:47:30.023604 1729563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:43789: Network error: Client connection negotiation failed: client connection to 127.152.231.252:43789: connect: Connection refused (error 111)
I20251027 02:47:30.031260 1729915 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:47:30.032773 1729887 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20251027 02:47:30.033264 1729887 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-2/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-2/wal
uuid: "947a379b7ea14817b0e501b4d845b6a3"
format_stamp: "Formatted at 2025-10-27 02:47:29 on 5fd53c4cbb9d"
I20251027 02:47:30.033735 1729887 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-2/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-2/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:47:30.081266 1729885 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } attempt: 7
W20251027 02:47:30.090878 1729885 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:43789: Network error: Client connection negotiation failed: client connection to 127.152.231.252:43789: connect: Connection refused (error 111)
I20251027 02:47:30.132625 1729887 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:47:30.134569 1729887 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:47:30.135267 1729887 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:47:30.178996 1729563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } attempt: 4
W20251027 02:47:30.184427 1729563 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:43789: Network error: Client connection negotiation failed: client connection to 127.152.231.252:43789: connect: Connection refused (error 111)
I20251027 02:47:30.257300 1729887 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.252:43789
I20251027 02:47:30.257455 1729968 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.252:43789 every 8 connection(s)
I20251027 02:47:30.261408 1729887 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-2/data/info.pb
I20251027 02:47:30.269766 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1729887
I20251027 02:47:30.270305 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/master-2/wal/instance
I20251027 02:47:30.271158 1729969 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:30.290228 1729969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:47:30.321818 1729969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:47:30.337177 1729969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:47:30.359160 1729969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3: Bootstrap starting.
I20251027 02:47:30.367871 1729969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:30.370019 1729969 log.cc:826] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3: Log is configured to *not* fsync() on all Append() calls
I20251027 02:47:30.376636 1729969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3: No bootstrap required, opened a new log
I20251027 02:47:30.404095 1729969 raft_consensus.cc:359] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } }
I20251027 02:47:30.405053 1729969 raft_consensus.cc:385] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:30.405421 1729969 raft_consensus.cc:740] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947a379b7ea14817b0e501b4d845b6a3, State: Initialized, Role: FOLLOWER
I20251027 02:47:30.406490 1729969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } }
I20251027 02:47:30.409960 1729977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } } }
I20251027 02:47:30.410869 1729977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:47:30.412670 1729969 sys_catalog.cc:565] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [sys.catalog]: configured and running, proceeding with master startup.
W20251027 02:47:30.462554 1729988 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251027 02:47:30.463572 1729988 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251027 02:47:30.445132 1729887 master_runner.cc:186] Error getting master registration for 127.152.231.254:35183: OK, instance_id { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" instance_seqno: 1761533245282004 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20251027 02:47:30.479735 1729887 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20251027 02:47:30.480883 1729563 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } attempt: 5
I20251027 02:47:30.503259 1729563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e: Bootstrap starting.
I20251027 02:47:30.510391 1729563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:30.513171 1729563 log.cc:826] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e: Log is configured to *not* fsync() on all Append() calls
I20251027 02:47:30.519739 1729563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e: No bootstrap required, opened a new log
I20251027 02:47:30.547462 1729563 raft_consensus.cc:359] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } }
I20251027 02:47:30.549119 1729563 raft_consensus.cc:385] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:30.549479 1729563 raft_consensus.cc:740] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4182f12cf5b4085b5f13f2a99304d3e, State: Initialized, Role: FOLLOWER
I20251027 02:47:30.550619 1729563 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } }
I20251027 02:47:30.555914 1729563 sys_catalog.cc:565] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e [sys.catalog]: configured and running, proceeding with master startup.
I20251027 02:47:30.557804 1729991 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } } }
I20251027 02:47:30.558784 1729991 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e [sys.catalog]: This master's current role is: FOLLOWER
W20251027 02:47:30.597175 1730002 catalog_manager.cc:1568] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251027 02:47:30.597640 1730002 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251027 02:47:30.593842 1729485 master_runner.cc:186] Error getting master registration for 127.152.231.253:40515: OK, instance_id { permanent_uuid: "736beccd865d4700879d4dde904dfb56" instance_seqno: 1761533247652277 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20251027 02:47:30.628340 1729485 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20251027 02:47:30.644981 1729923 tablet.cc:2378] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20251027 02:47:30.734285 1729517 tablet.cc:2378] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:47:30.812681 1729977 raft_consensus.cc:493] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:47:30.813328 1729977 raft_consensus.cc:515] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } }
I20251027 02:47:30.816337 1729977 leader_election.cc:290] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4182f12cf5b4085b5f13f2a99304d3e (127.152.231.254:35183), 736beccd865d4700879d4dde904dfb56 (127.152.231.253:40515)
I20251027 02:47:30.821380 1729538 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "947a379b7ea14817b0e501b4d845b6a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" is_pre_election: true
I20251027 02:47:30.822609 1729538 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 947a379b7ea14817b0e501b4d845b6a3 in term 0.
I20251027 02:47:30.824177 1729919 leader_election.cc:304] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947a379b7ea14817b0e501b4d845b6a3, c4182f12cf5b4085b5f13f2a99304d3e; no voters:
I20251027 02:47:30.826297 1729977 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:47:30.826910 1729977 raft_consensus.cc:493] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:47:30.827703 1729977 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:30.840912 1729977 raft_consensus.cc:515] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } }
I20251027 02:47:30.845281 1729538 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "947a379b7ea14817b0e501b4d845b6a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4182f12cf5b4085b5f13f2a99304d3e"
I20251027 02:47:30.845889 1729538 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:30.853469 1729538 raft_consensus.cc:2468] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947a379b7ea14817b0e501b4d845b6a3 in term 1.
I20251027 02:47:30.855057 1729919 leader_election.cc:304] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947a379b7ea14817b0e501b4d845b6a3, c4182f12cf5b4085b5f13f2a99304d3e; no voters:
I20251027 02:47:30.858363 1729977 leader_election.cc:290] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [CANDIDATE]: Term 1 election: Requested vote from peers c4182f12cf5b4085b5f13f2a99304d3e (127.152.231.254:35183), 736beccd865d4700879d4dde904dfb56 (127.152.231.253:40515)
I20251027 02:47:30.858796 1730004 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:47:30.859380 1730004 raft_consensus.cc:697] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [term 1 LEADER]: Becoming Leader. State: Replica: 947a379b7ea14817b0e501b4d845b6a3, State: Running, Role: LEADER
I20251027 02:47:30.860404 1730004 consensus_queue.cc:237] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } }
I20251027 02:47:30.872007 1729977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 947a379b7ea14817b0e501b4d845b6a3. Latest consensus state: current_term: 1 leader_uuid: "947a379b7ea14817b0e501b4d845b6a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } } }
I20251027 02:47:30.872723 1729977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [sys.catalog]: This master's current role is: LEADER
I20251027 02:47:30.878261 1730006 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251027 02:47:30.904419 1730006 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251027 02:47:30.923265 1729538 raft_consensus.cc:1275] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e [term 1 FOLLOWER]: Refusing update from remote peer 947a379b7ea14817b0e501b4d845b6a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:47:30.925776 1729977 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251027 02:47:30.947156 1729991 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 947a379b7ea14817b0e501b4d845b6a3. Latest consensus state: current_term: 1 leader_uuid: "947a379b7ea14817b0e501b4d845b6a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } } }
I20251027 02:47:30.947880 1729991 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:47:30.948905 1730004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "947a379b7ea14817b0e501b4d845b6a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } } }
I20251027 02:47:30.949605 1730004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [sys.catalog]: This master's current role is: LEADER
I20251027 02:47:30.954571 1730006 catalog_manager.cc:1357] Generated new cluster ID: 4179ec475e9d40438460af941327f81f
I20251027 02:47:30.954895 1730006 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251027 02:47:30.955809 1729991 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "947a379b7ea14817b0e501b4d845b6a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } } }
I20251027 02:47:30.956525 1729991 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:47:31.006193 1730006 catalog_manager.cc:1380] Generated new certificate authority record
I20251027 02:47:31.008463 1730006 catalog_manager.cc:1514] Loading token signing keys...
I20251027 02:47:31.035249 1730006 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3: Generated new TSK 0
I20251027 02:47:31.036535 1730006 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251027 02:47:31.063822 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:47:31.066659 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.152.231.193:0
--local_ip_for_outbound_sockets=127.152.231.193
--webserver_interface=127.152.231.193
--webserver_port=0
--tserver_master_addrs=127.152.231.254:35183,127.152.231.253:40515,127.152.231.252:43789
--builtin_ntp_servers=127.152.231.212:43561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www with env {}
W20251027 02:47:31.585639 1730012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:47:31.586453 1730012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:47:31.587266 1730012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20251027 02:47:31.603565 1730002 catalog_manager.cc:1269] Loaded cluster ID: 4179ec475e9d40438460af941327f81f
I20251027 02:47:31.603852 1730002 catalog_manager.cc:1562] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e: loading cluster ID for follower catalog manager: success
I20251027 02:47:31.610669 1730002 catalog_manager.cc:1584] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e: acquiring CA information for follower catalog manager: success
I20251027 02:47:31.615873 1730002 catalog_manager.cc:1612] T 00000000000000000000000000000000 P c4182f12cf5b4085b5f13f2a99304d3e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20251027 02:47:31.641822 1730012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:47:31.643097 1730012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.152.231.193
I20251027 02:47:31.701556 1730012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:43561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.152.231.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-0/data/info.pb
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.193
--webserver_port=0
--tserver_master_addrs=127.152.231.254:35183,127.152.231.253:40515,127.152.231.252:43789
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.152.231.193
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:47:31.703397 1730012 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:47:31.705772 1730012 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:47:31.724700 1730018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:47:32.181311 1729885 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } attempt: 8
I20251027 02:47:32.207926 1729885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56: Bootstrap starting.
I20251027 02:47:32.215281 1729885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:32.217456 1729885 log.cc:826] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56: Log is configured to *not* fsync() on all Append() calls
I20251027 02:47:32.229827 1729885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56: No bootstrap required, opened a new log
I20251027 02:47:32.274820 1729885 raft_consensus.cc:359] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } }
I20251027 02:47:32.276377 1729885 raft_consensus.cc:385] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:32.276953 1729885 raft_consensus.cc:740] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 736beccd865d4700879d4dde904dfb56, State: Initialized, Role: FOLLOWER
I20251027 02:47:32.278785 1729885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } }
I20251027 02:47:32.294600 1730048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } } }
I20251027 02:47:32.299906 1729856 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:32.298318 1729857 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "947a379b7ea14817b0e501b4d845b6a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736beccd865d4700879d4dde904dfb56"
I20251027 02:47:32.298368 1729858 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "947a379b7ea14817b0e501b4d845b6a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736beccd865d4700879d4dde904dfb56" is_pre_election: true
I20251027 02:47:32.300405 1729885 sys_catalog.cc:565] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56 [sys.catalog]: configured and running, proceeding with master startup.
W20251027 02:47:32.318408 1729858 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56854 (request call id 1) took 1499 ms (client timeout 1925 ms). Trace:
W20251027 02:47:32.318472 1729857 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56854 (request call id 2) took 1459 ms (client timeout 1916 ms). Trace:
W20251027 02:47:32.318848 1729858 rpcz_store.cc:269] 1027 02:47:30.819001 (+ 0us) service_pool.cc:168] Inserting onto call queue
1027 02:47:30.819171 (+ 170us) service_pool.cc:225] Handling call
1027 02:47:32.318316 (+1499145us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20251027 02:47:32.319015 1729857 rpcz_store.cc:269] 1027 02:47:30.858639 (+ 0us) service_pool.cc:168] Inserting onto call queue
1027 02:47:30.858800 (+ 161us) service_pool.cc:225] Handling call
1027 02:47:32.318384 (+1459584us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20251027 02:47:32.322710 1729856 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56 [term 1 FOLLOWER]: Refusing update from remote peer 947a379b7ea14817b0e501b4d845b6a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:47:32.323886 1729856 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56854 (request call id 3) took 1402 ms. Trace:
I20251027 02:47:32.324389 1729856 rpcz_store.cc:276] 1027 02:47:30.921932 (+ 0us) service_pool.cc:168] Inserting onto call queue
1027 02:47:30.922116 (+ 184us) service_pool.cc:225] Handling call
1027 02:47:32.299057 (+1376941us) raft_consensus.cc:1471] Updating replica for 0 ops
1027 02:47:32.323329 (+ 24272us) raft_consensus.cc:1697] Filling consensus response to leader.
1027 02:47:32.323790 (+ 461us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20251027 02:47:32.325546 1730048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:47:32.327016 1730028 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251027 02:47:32.328292 1729837 tablet.cc:2378] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:47:32.411643 1730048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 947a379b7ea14817b0e501b4d845b6a3. Latest consensus state: current_term: 1 leader_uuid: "947a379b7ea14817b0e501b4d845b6a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } } }
I20251027 02:47:32.412418 1730048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:47:32.413421 1730058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "947a379b7ea14817b0e501b4d845b6a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } } }
I20251027 02:47:32.415820 1730058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:47:32.443519 1730067 catalog_manager.cc:1269] Loaded cluster ID: 4179ec475e9d40438460af941327f81f
I20251027 02:47:32.443890 1730067 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56: loading cluster ID for follower catalog manager: success
I20251027 02:47:32.452620 1730067 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56: acquiring CA information for follower catalog manager: success
I20251027 02:47:32.458096 1730067 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 736beccd865d4700879d4dde904dfb56: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251027 02:47:32.822372 1730028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "947a379b7ea14817b0e501b4d845b6a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4182f12cf5b4085b5f13f2a99304d3e" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 35183 } } peers { permanent_uuid: "736beccd865d4700879d4dde904dfb56" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 40515 } } peers { permanent_uuid: "947a379b7ea14817b0e501b4d845b6a3" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 43789 } } }
I20251027 02:47:32.823056 1730028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 947a379b7ea14817b0e501b4d845b6a3 [sys.catalog]: This master's current role is: LEADER
W20251027 02:47:33.125602 1730017 debug-util.cc:398] Leaking SignalData structure 0x7b0800037940 after lost signal to thread 1730012
W20251027 02:47:31.725641 1730019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:47:33.214141 1730012 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.489s user 0.650s sys 0.801s
W20251027 02:47:33.214562 1730012 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.489s user 0.650s sys 0.801s
W20251027 02:47:33.217586 1730021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:47:33.223435 1730020 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1493 milliseconds
I20251027 02:47:33.223532 1730012 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:47:33.225035 1730012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:47:33.228850 1730012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:47:33.230343 1730012 hybrid_clock.cc:648] HybridClock initialized: now 1761533253230250 us; error 85 us; skew 500 ppm
I20251027 02:47:33.231489 1730012 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:47:33.239497 1730012 webserver.cc:492] Webserver started at http://127.152.231.193:42599/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:47:33.240764 1730012 fs_manager.cc:362] Metadata directory not provided
I20251027 02:47:33.241032 1730012 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:47:33.241674 1730012 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:47:33.249995 1730012 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-0/data/instance:
uuid: "e48d138f3a134c448081b884d795716b"
format_stamp: "Formatted at 2025-10-27 02:47:33 on 5fd53c4cbb9d"
I20251027 02:47:33.257866 1730012 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-0/wal/instance:
uuid: "e48d138f3a134c448081b884d795716b"
format_stamp: "Formatted at 2025-10-27 02:47:33 on 5fd53c4cbb9d"
I20251027 02:47:33.269294 1730012 fs_manager.cc:696] Time spent creating directory manager: real 0.010s user 0.011s sys 0.000s
I20251027 02:47:33.277985 1730081 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:47:33.279486 1730012 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20251027 02:47:33.279915 1730012 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-0/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-0/wal
uuid: "e48d138f3a134c448081b884d795716b"
format_stamp: "Formatted at 2025-10-27 02:47:33 on 5fd53c4cbb9d"
I20251027 02:47:33.280405 1730012 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:47:33.390836 1730012 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:47:33.392690 1730012 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:47:33.393350 1730012 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:47:33.397056 1730012 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251027 02:47:33.404093 1730012 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251027 02:47:33.404484 1730012 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20251027 02:47:33.404855 1730012 ts_tablet_manager.cc:616] Registered 0 tablets
I20251027 02:47:33.405292 1730012 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251027 02:47:33.641494 1730012 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.193:44051
I20251027 02:47:33.641654 1730195 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.193:44051 every 8 connection(s)
I20251027 02:47:33.645340 1730012 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-0/data/info.pb
I20251027 02:47:33.647351 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1730012
I20251027 02:47:33.647889 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-0/wal/instance
I20251027 02:47:33.665277 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:47:33.667403 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.152.231.194:0
--local_ip_for_outbound_sockets=127.152.231.194
--webserver_interface=127.152.231.194
--webserver_port=0
--tserver_master_addrs=127.152.231.254:35183,127.152.231.253:40515,127.152.231.252:43789
--builtin_ntp_servers=127.152.231.212:43561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www with env {}
I20251027 02:47:33.687855 1730198 heartbeater.cc:344] Connected to a master server at 127.152.231.254:35183
I20251027 02:47:33.688434 1730198 heartbeater.cc:461] Registering TS with master...
I20251027 02:47:33.689990 1730198 heartbeater.cc:507] Master 127.152.231.254:35183 requested a full tablet report, sending...
I20251027 02:47:33.690127 1730196 heartbeater.cc:344] Connected to a master server at 127.152.231.252:43789
I20251027 02:47:33.690487 1730197 heartbeater.cc:344] Connected to a master server at 127.152.231.253:40515
I20251027 02:47:33.690656 1730196 heartbeater.cc:461] Registering TS with master...
I20251027 02:47:33.690842 1730197 heartbeater.cc:461] Registering TS with master...
I20251027 02:47:33.691967 1730197 heartbeater.cc:507] Master 127.152.231.253:40515 requested a full tablet report, sending...
I20251027 02:47:33.692044 1730196 heartbeater.cc:507] Master 127.152.231.252:43789 requested a full tablet report, sending...
I20251027 02:47:33.694528 1729528 ts_manager.cc:194] Registered new tserver with Master: e48d138f3a134c448081b884d795716b (127.152.231.193:44051)
I20251027 02:47:33.694980 1729934 ts_manager.cc:194] Registered new tserver with Master: e48d138f3a134c448081b884d795716b (127.152.231.193:44051)
I20251027 02:47:33.695858 1729848 ts_manager.cc:194] Registered new tserver with Master: e48d138f3a134c448081b884d795716b (127.152.231.193:44051)
I20251027 02:47:33.697741 1729934 master_service.cc:502] Signed X509 certificate for tserver {username='jenkins-slave'} at 127.152.231.193:37719
W20251027 02:47:34.172863 1730206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:47:34.173611 1730206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:47:34.174454 1730206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:47:34.231683 1730206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:47:34.233151 1730206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.152.231.194
I20251027 02:47:34.297245 1730206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:43561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.152.231.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-1/data/info.pb
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.194
--webserver_port=0
--tserver_master_addrs=127.152.231.254:35183,127.152.231.253:40515,127.152.231.252:43789
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.152.231.194
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:47:34.299191 1730206 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:47:34.301652 1730206 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:47:34.321215 1730213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:47:34.702013 1730196 heartbeater.cc:499] Master 127.152.231.252:43789 was elected leader, sending a full tablet report...
W20251027 02:47:35.721722 1730212 debug-util.cc:398] Leaking SignalData structure 0x7b0800037940 after lost signal to thread 1730206
W20251027 02:47:34.321808 1730214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:47:35.828141 1730206 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.507s user 0.647s sys 0.858s
W20251027 02:47:35.828614 1730206 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.507s user 0.647s sys 0.858s
W20251027 02:47:35.831457 1730216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:47:35.833499 1730215 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1506 milliseconds
I20251027 02:47:35.833595 1730206 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:47:35.835167 1730206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:47:35.838780 1730206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:47:35.840328 1730206 hybrid_clock.cc:648] HybridClock initialized: now 1761533255840274 us; error 32 us; skew 500 ppm
I20251027 02:47:35.841555 1730206 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:47:35.849649 1730206 webserver.cc:492] Webserver started at http://127.152.231.194:42311/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:47:35.850972 1730206 fs_manager.cc:362] Metadata directory not provided
I20251027 02:47:35.851444 1730206 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:47:35.852154 1730206 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:47:35.860076 1730206 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-1/data/instance:
uuid: "e5e7b8aa196948a8aab515b05ee9e95e"
format_stamp: "Formatted at 2025-10-27 02:47:35 on 5fd53c4cbb9d"
I20251027 02:47:35.861792 1730206 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-1/wal/instance:
uuid: "e5e7b8aa196948a8aab515b05ee9e95e"
format_stamp: "Formatted at 2025-10-27 02:47:35 on 5fd53c4cbb9d"
I20251027 02:47:35.873119 1730206 fs_manager.cc:696] Time spent creating directory manager: real 0.010s user 0.009s sys 0.001s
I20251027 02:47:35.881630 1730234 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:47:35.882987 1730206 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
I20251027 02:47:35.883397 1730206 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-1/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-1/wal
uuid: "e5e7b8aa196948a8aab515b05ee9e95e"
format_stamp: "Formatted at 2025-10-27 02:47:35 on 5fd53c4cbb9d"
I20251027 02:47:35.883817 1730206 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:47:35.998075 1730206 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:47:35.999922 1730206 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:47:36.000619 1730206 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:47:36.004341 1730206 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251027 02:47:36.011749 1730206 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251027 02:47:36.012046 1730206 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251027 02:47:36.012351 1730206 ts_tablet_manager.cc:616] Registered 0 tablets
I20251027 02:47:36.012562 1730206 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251027 02:47:36.245580 1730206 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.194:35939
I20251027 02:47:36.245728 1730346 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.194:35939 every 8 connection(s)
I20251027 02:47:36.249311 1730206 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-1/data/info.pb
I20251027 02:47:36.256973 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1730206
I20251027 02:47:36.257515 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-1/wal/instance
I20251027 02:47:36.266067 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:47:36.268096 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.152.231.195:0
--local_ip_for_outbound_sockets=127.152.231.195
--webserver_interface=127.152.231.195
--webserver_port=0
--tserver_master_addrs=127.152.231.254:35183,127.152.231.253:40515,127.152.231.252:43789
--builtin_ntp_servers=127.152.231.212:43561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www with env {}
I20251027 02:47:36.288494 1730347 heartbeater.cc:344] Connected to a master server at 127.152.231.252:43789
I20251027 02:47:36.289055 1730347 heartbeater.cc:461] Registering TS with master...
I20251027 02:47:36.289461 1730349 heartbeater.cc:344] Connected to a master server at 127.152.231.254:35183
I20251027 02:47:36.289876 1730348 heartbeater.cc:344] Connected to a master server at 127.152.231.253:40515
I20251027 02:47:36.290109 1730349 heartbeater.cc:461] Registering TS with master...
I20251027 02:47:36.290251 1730348 heartbeater.cc:461] Registering TS with master...
I20251027 02:47:36.290774 1730347 heartbeater.cc:507] Master 127.152.231.252:43789 requested a full tablet report, sending...
I20251027 02:47:36.291473 1730349 heartbeater.cc:507] Master 127.152.231.254:35183 requested a full tablet report, sending...
I20251027 02:47:36.291486 1730348 heartbeater.cc:507] Master 127.152.231.253:40515 requested a full tablet report, sending...
I20251027 02:47:36.293648 1729934 ts_manager.cc:194] Registered new tserver with Master: e5e7b8aa196948a8aab515b05ee9e95e (127.152.231.194:35939)
I20251027 02:47:36.294979 1729528 ts_manager.cc:194] Registered new tserver with Master: e5e7b8aa196948a8aab515b05ee9e95e (127.152.231.194:35939)
I20251027 02:47:36.294993 1729848 ts_manager.cc:194] Registered new tserver with Master: e5e7b8aa196948a8aab515b05ee9e95e (127.152.231.194:35939)
I20251027 02:47:36.295301 1729934 master_service.cc:502] Signed X509 certificate for tserver {username='jenkins-slave'} at 127.152.231.194:56517
W20251027 02:47:36.806176 1730357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:47:36.806911 1730357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:47:36.807863 1730357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:47:36.869400 1730357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:47:36.870983 1730357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.152.231.195
I20251027 02:47:36.939791 1730357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:43561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.152.231.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-2/data/info.pb
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.195
--webserver_port=0
--tserver_master_addrs=127.152.231.254:35183,127.152.231.253:40515,127.152.231.252:43789
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.152.231.195
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:47:36.941844 1730357 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:47:36.944466 1730357 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:47:36.964465 1730375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:47:37.299142 1730347 heartbeater.cc:499] Master 127.152.231.252:43789 was elected leader, sending a full tablet report...
W20251027 02:47:36.965396 1730376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:47:38.117456 1730378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:47:38.119930 1730377 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1148 milliseconds
W20251027 02:47:38.120512 1730357 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.156s user 0.497s sys 0.657s
W20251027 02:47:38.120820 1730357 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.156s user 0.497s sys 0.657s
I20251027 02:47:38.121060 1730357 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:47:38.122499 1730357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:47:38.126034 1730357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:47:38.127481 1730357 hybrid_clock.cc:648] HybridClock initialized: now 1761533258127422 us; error 46 us; skew 500 ppm
I20251027 02:47:38.128578 1730357 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:47:38.137532 1730357 webserver.cc:492] Webserver started at http://127.152.231.195:38667/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:47:38.138857 1730357 fs_manager.cc:362] Metadata directory not provided
I20251027 02:47:38.139155 1730357 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:47:38.139746 1730357 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:47:38.147735 1730357 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-2/data/instance:
uuid: "a727b6e247b14eb59af28df77d7afa22"
format_stamp: "Formatted at 2025-10-27 02:47:38 on 5fd53c4cbb9d"
I20251027 02:47:38.149330 1730357 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-2/wal/instance:
uuid: "a727b6e247b14eb59af28df77d7afa22"
format_stamp: "Formatted at 2025-10-27 02:47:38 on 5fd53c4cbb9d"
I20251027 02:47:38.159974 1730357 fs_manager.cc:696] Time spent creating directory manager: real 0.010s user 0.001s sys 0.009s
I20251027 02:47:38.167618 1730412 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:47:38.169010 1730357 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20251027 02:47:38.169431 1730357 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-2/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-2/wal
uuid: "a727b6e247b14eb59af28df77d7afa22"
format_stamp: "Formatted at 2025-10-27 02:47:38 on 5fd53c4cbb9d"
I20251027 02:47:38.169842 1730357 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:47:38.282238 1730357 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:47:38.284085 1730357 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:47:38.284691 1730357 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:47:38.288877 1730357 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251027 02:47:38.296240 1730357 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251027 02:47:38.296509 1730357 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251027 02:47:38.296782 1730357 ts_tablet_manager.cc:616] Registered 0 tablets
I20251027 02:47:38.296985 1730357 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251027 02:47:38.509991 1730357 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.195:42803
I20251027 02:47:38.510072 1730540 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.195:42803 every 8 connection(s)
I20251027 02:47:38.512691 1730357 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-2/data/info.pb
I20251027 02:47:38.516026 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1730357
I20251027 02:47:38.516587 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_0.1761533243417696-1729439-0/minicluster-data/ts-2/wal/instance
I20251027 02:47:38.543022 1730544 heartbeater.cc:344] Connected to a master server at 127.152.231.253:40515
I20251027 02:47:38.543555 1730544 heartbeater.cc:461] Registering TS with master...
I20251027 02:47:38.544426 1730545 heartbeater.cc:344] Connected to a master server at 127.152.231.254:35183
I20251027 02:47:38.544894 1730545 heartbeater.cc:461] Registering TS with master...
I20251027 02:47:38.545112 1730544 heartbeater.cc:507] Master 127.152.231.253:40515 requested a full tablet report, sending...
I20251027 02:47:38.545383 1730543 heartbeater.cc:344] Connected to a master server at 127.152.231.252:43789
I20251027 02:47:38.545727 1730543 heartbeater.cc:461] Registering TS with master...
I20251027 02:47:38.546176 1730545 heartbeater.cc:507] Master 127.152.231.254:35183 requested a full tablet report, sending...
I20251027 02:47:38.546540 1730543 heartbeater.cc:507] Master 127.152.231.252:43789 requested a full tablet report, sending...
I20251027 02:47:38.548267 1729848 ts_manager.cc:194] Registered new tserver with Master: a727b6e247b14eb59af28df77d7afa22 (127.152.231.195:42803)
I20251027 02:47:38.548734 1729528 ts_manager.cc:194] Registered new tserver with Master: a727b6e247b14eb59af28df77d7afa22 (127.152.231.195:42803)
I20251027 02:47:38.549629 1729934 ts_manager.cc:194] Registered new tserver with Master: a727b6e247b14eb59af28df77d7afa22 (127.152.231.195:42803)
I20251027 02:47:38.551378 1729934 master_service.cc:502] Signed X509 certificate for tserver {username='jenkins-slave'} at 127.152.231.195:47833
I20251027 02:47:38.558421 1729439 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251027 02:47:38.568020 1729439 test_util.cc:276] Using random seed: -333246152
I20251027 02:47:38.614779 1729933 catalog_manager.cc:2257] Servicing CreateTable request from {username='jenkins-slave'} at 127.0.0.1:39492:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\004\001\000UUU\025\004\001\000\252\252\252*\004\001\000\377\377\377?\004\001\000TUUU\004\001\000\251\252\252j"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251027 02:47:38.618654 1729933 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251027 02:47:38.701005 1730471 tablet_service.cc:1505] Processing CreateTablet for tablet 56a1f0cb4ab34f2fb12c1fadfea103b7 (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20251027 02:47:38.700243 1730470 tablet_service.cc:1505] Processing CreateTablet for tablet 932afee710bd49159618bd88c0295333 (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20251027 02:47:38.700886 1730469 tablet_service.cc:1505] Processing CreateTablet for tablet 028cac69bd2c4adda9da6b2e07fcee84 (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20251027 02:47:38.700273 1730474 tablet_service.cc:1505] Processing CreateTablet for tablet ee405edc42594078b91898ef9fbf8b33 (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION VALUES < 357913941
I20251027 02:47:38.700208 1730472 tablet_service.cc:1505] Processing CreateTablet for tablet 3b4653bd3b624448801635ef21288354 (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20251027 02:47:38.701113 1730473 tablet_service.cc:1505] Processing CreateTablet for tablet 1a5f4ca8542948ce9886a384b1f97b3a (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20251027 02:47:38.705377 1730469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 028cac69bd2c4adda9da6b2e07fcee84. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.706009 1730472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b4653bd3b624448801635ef21288354. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.706542 1730474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee405edc42594078b91898ef9fbf8b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.707038 1730470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 932afee710bd49159618bd88c0295333. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.707811 1730473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a5f4ca8542948ce9886a384b1f97b3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.708479 1730471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a1f0cb4ab34f2fb12c1fadfea103b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.732725 1730579 tablet_bootstrap.cc:492] T 028cac69bd2c4adda9da6b2e07fcee84 P a727b6e247b14eb59af28df77d7afa22: Bootstrap starting.
I20251027 02:47:38.739672 1730277 tablet_service.cc:1505] Processing CreateTablet for tablet 028cac69bd2c4adda9da6b2e07fcee84 (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20251027 02:47:38.739209 1730282 tablet_service.cc:1505] Processing CreateTablet for tablet ee405edc42594078b91898ef9fbf8b33 (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION VALUES < 357913941
I20251027 02:47:38.739180 1730279 tablet_service.cc:1505] Processing CreateTablet for tablet 56a1f0cb4ab34f2fb12c1fadfea103b7 (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20251027 02:47:38.739166 1730281 tablet_service.cc:1505] Processing CreateTablet for tablet 1a5f4ca8542948ce9886a384b1f97b3a (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20251027 02:47:38.740739 1730579 tablet_bootstrap.cc:654] T 028cac69bd2c4adda9da6b2e07fcee84 P a727b6e247b14eb59af28df77d7afa22: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.739667 1730278 tablet_service.cc:1505] Processing CreateTablet for tablet 932afee710bd49159618bd88c0295333 (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20251027 02:47:38.738467 1730280 tablet_service.cc:1505] Processing CreateTablet for tablet 3b4653bd3b624448801635ef21288354 (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20251027 02:47:38.741247 1730127 tablet_service.cc:1505] Processing CreateTablet for tablet 56a1f0cb4ab34f2fb12c1fadfea103b7 (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20251027 02:47:38.742523 1730278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 932afee710bd49159618bd88c0295333. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.742714 1730579 log.cc:826] T 028cac69bd2c4adda9da6b2e07fcee84 P a727b6e247b14eb59af28df77d7afa22: Log is configured to *not* fsync() on all Append() calls
I20251027 02:47:38.741252 1730125 tablet_service.cc:1505] Processing CreateTablet for tablet 028cac69bd2c4adda9da6b2e07fcee84 (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20251027 02:47:38.743160 1730282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee405edc42594078b91898ef9fbf8b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.743162 1730127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a1f0cb4ab34f2fb12c1fadfea103b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.743696 1730280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b4653bd3b624448801635ef21288354. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.743868 1730125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 028cac69bd2c4adda9da6b2e07fcee84. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.744335 1730281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a5f4ca8542948ce9886a384b1f97b3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.745039 1730277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 028cac69bd2c4adda9da6b2e07fcee84. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.745611 1730279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a1f0cb4ab34f2fb12c1fadfea103b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.748994 1730579 tablet_bootstrap.cc:492] T 028cac69bd2c4adda9da6b2e07fcee84 P a727b6e247b14eb59af28df77d7afa22: No bootstrap required, opened a new log
I20251027 02:47:38.749505 1730579 ts_tablet_manager.cc:1403] T 028cac69bd2c4adda9da6b2e07fcee84 P a727b6e247b14eb59af28df77d7afa22: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.006s
I20251027 02:47:38.751684 1730129 tablet_service.cc:1505] Processing CreateTablet for tablet 1a5f4ca8542948ce9886a384b1f97b3a (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20251027 02:47:38.751483 1730128 tablet_service.cc:1505] Processing CreateTablet for tablet 3b4653bd3b624448801635ef21288354 (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20251027 02:47:38.751788 1730130 tablet_service.cc:1505] Processing CreateTablet for tablet ee405edc42594078b91898ef9fbf8b33 (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION VALUES < 357913941
I20251027 02:47:38.751844 1730126 tablet_service.cc:1505] Processing CreateTablet for tablet 932afee710bd49159618bd88c0295333 (DEFAULT_TABLE table=test-workload [id=a62c2aa9a80d46b990c6068782281e6d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20251027 02:47:38.753453 1730129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a5f4ca8542948ce9886a384b1f97b3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.754179 1730128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b4653bd3b624448801635ef21288354. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.754801 1730126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 932afee710bd49159618bd88c0295333. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.755421 1730130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee405edc42594078b91898ef9fbf8b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:47:38.770773 1730581 tablet_bootstrap.cc:492] T 3b4653bd3b624448801635ef21288354 P e48d138f3a134c448081b884d795716b: Bootstrap starting.
I20251027 02:47:38.771713 1730582 tablet_bootstrap.cc:492] T 028cac69bd2c4adda9da6b2e07fcee84 P e5e7b8aa196948a8aab515b05ee9e95e: Bootstrap starting.
I20251027 02:47:38.772976 1730579 raft_consensus.cc:359] T 028cac69bd2c4adda9da6b2e07fcee84 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.774446 1730579 raft_consensus.cc:385] T 028cac69bd2c4adda9da6b2e07fcee84 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.774698 1730579 raft_consensus.cc:740] T 028cac69bd2c4adda9da6b2e07fcee84 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a727b6e247b14eb59af28df77d7afa22, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.775470 1730579 consensus_queue.cc:260] T 028cac69bd2c4adda9da6b2e07fcee84 P a727b6e247b14eb59af28df77d7afa22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.778529 1730543 heartbeater.cc:499] Master 127.152.231.252:43789 was elected leader, sending a full tablet report...
I20251027 02:47:38.779095 1730579 ts_tablet_manager.cc:1434] T 028cac69bd2c4adda9da6b2e07fcee84 P a727b6e247b14eb59af28df77d7afa22: Time spent starting tablet: real 0.029s user 0.019s sys 0.009s
I20251027 02:47:38.780038 1730579 tablet_bootstrap.cc:492] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22: Bootstrap starting.
I20251027 02:47:38.780356 1730581 tablet_bootstrap.cc:654] T 3b4653bd3b624448801635ef21288354 P e48d138f3a134c448081b884d795716b: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.780530 1730582 tablet_bootstrap.cc:654] T 028cac69bd2c4adda9da6b2e07fcee84 P e5e7b8aa196948a8aab515b05ee9e95e: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.782423 1730582 log.cc:826] T 028cac69bd2c4adda9da6b2e07fcee84 P e5e7b8aa196948a8aab515b05ee9e95e: Log is configured to *not* fsync() on all Append() calls
I20251027 02:47:38.782471 1730581 log.cc:826] T 3b4653bd3b624448801635ef21288354 P e48d138f3a134c448081b884d795716b: Log is configured to *not* fsync() on all Append() calls
I20251027 02:47:38.784852 1730579 tablet_bootstrap.cc:654] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.788393 1730582 tablet_bootstrap.cc:492] T 028cac69bd2c4adda9da6b2e07fcee84 P e5e7b8aa196948a8aab515b05ee9e95e: No bootstrap required, opened a new log
I20251027 02:47:38.788890 1730582 ts_tablet_manager.cc:1403] T 028cac69bd2c4adda9da6b2e07fcee84 P e5e7b8aa196948a8aab515b05ee9e95e: Time spent bootstrapping tablet: real 0.018s user 0.014s sys 0.000s
I20251027 02:47:38.789321 1730581 tablet_bootstrap.cc:492] T 3b4653bd3b624448801635ef21288354 P e48d138f3a134c448081b884d795716b: No bootstrap required, opened a new log
I20251027 02:47:38.789834 1730581 ts_tablet_manager.cc:1403] T 3b4653bd3b624448801635ef21288354 P e48d138f3a134c448081b884d795716b: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.010s
I20251027 02:47:38.790781 1730579 tablet_bootstrap.cc:492] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22: No bootstrap required, opened a new log
I20251027 02:47:38.791112 1730579 ts_tablet_manager.cc:1403] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20251027 02:47:38.793138 1730579 raft_consensus.cc:359] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } }
I20251027 02:47:38.793707 1730579 raft_consensus.cc:385] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.793921 1730579 raft_consensus.cc:740] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a727b6e247b14eb59af28df77d7afa22, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.794430 1730579 consensus_queue.cc:260] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } }
I20251027 02:47:38.796068 1730579 ts_tablet_manager.cc:1434] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20251027 02:47:38.796746 1730579 tablet_bootstrap.cc:492] T 932afee710bd49159618bd88c0295333 P a727b6e247b14eb59af28df77d7afa22: Bootstrap starting.
I20251027 02:47:38.802162 1730579 tablet_bootstrap.cc:654] T 932afee710bd49159618bd88c0295333 P a727b6e247b14eb59af28df77d7afa22: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.807886 1730579 tablet_bootstrap.cc:492] T 932afee710bd49159618bd88c0295333 P a727b6e247b14eb59af28df77d7afa22: No bootstrap required, opened a new log
I20251027 02:47:38.808238 1730579 ts_tablet_manager.cc:1403] T 932afee710bd49159618bd88c0295333 P a727b6e247b14eb59af28df77d7afa22: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.007s
I20251027 02:47:38.810256 1730579 raft_consensus.cc:359] T 932afee710bd49159618bd88c0295333 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.810791 1730579 raft_consensus.cc:385] T 932afee710bd49159618bd88c0295333 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.811005 1730579 raft_consensus.cc:740] T 932afee710bd49159618bd88c0295333 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a727b6e247b14eb59af28df77d7afa22, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.811503 1730579 consensus_queue.cc:260] T 932afee710bd49159618bd88c0295333 P a727b6e247b14eb59af28df77d7afa22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.811717 1730582 raft_consensus.cc:359] T 028cac69bd2c4adda9da6b2e07fcee84 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.812538 1730582 raft_consensus.cc:385] T 028cac69bd2c4adda9da6b2e07fcee84 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.812790 1730582 raft_consensus.cc:740] T 028cac69bd2c4adda9da6b2e07fcee84 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5e7b8aa196948a8aab515b05ee9e95e, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.813248 1730579 ts_tablet_manager.cc:1434] T 932afee710bd49159618bd88c0295333 P a727b6e247b14eb59af28df77d7afa22: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20251027 02:47:38.813601 1730582 consensus_queue.cc:260] T 028cac69bd2c4adda9da6b2e07fcee84 P e5e7b8aa196948a8aab515b05ee9e95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.814067 1730579 tablet_bootstrap.cc:492] T 3b4653bd3b624448801635ef21288354 P a727b6e247b14eb59af28df77d7afa22: Bootstrap starting.
I20251027 02:47:38.814483 1730581 raft_consensus.cc:359] T 3b4653bd3b624448801635ef21288354 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.815285 1730581 raft_consensus.cc:385] T 3b4653bd3b624448801635ef21288354 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.815544 1730581 raft_consensus.cc:740] T 3b4653bd3b624448801635ef21288354 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e48d138f3a134c448081b884d795716b, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.816334 1730581 consensus_queue.cc:260] T 3b4653bd3b624448801635ef21288354 P e48d138f3a134c448081b884d795716b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.816826 1730582 ts_tablet_manager.cc:1434] T 028cac69bd2c4adda9da6b2e07fcee84 P e5e7b8aa196948a8aab515b05ee9e95e: Time spent starting tablet: real 0.028s user 0.026s sys 0.000s
I20251027 02:47:38.817827 1730582 tablet_bootstrap.cc:492] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e: Bootstrap starting.
I20251027 02:47:38.818960 1730579 tablet_bootstrap.cc:654] T 3b4653bd3b624448801635ef21288354 P a727b6e247b14eb59af28df77d7afa22: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.819818 1730581 ts_tablet_manager.cc:1434] T 3b4653bd3b624448801635ef21288354 P e48d138f3a134c448081b884d795716b: Time spent starting tablet: real 0.030s user 0.028s sys 0.001s
I20251027 02:47:38.820781 1730581 tablet_bootstrap.cc:492] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b: Bootstrap starting.
I20251027 02:47:38.822034 1730582 tablet_bootstrap.cc:654] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.825583 1730579 tablet_bootstrap.cc:492] T 3b4653bd3b624448801635ef21288354 P a727b6e247b14eb59af28df77d7afa22: No bootstrap required, opened a new log
I20251027 02:47:38.825604 1730581 tablet_bootstrap.cc:654] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.825947 1730579 ts_tablet_manager.cc:1403] T 3b4653bd3b624448801635ef21288354 P a727b6e247b14eb59af28df77d7afa22: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20251027 02:47:38.827904 1730579 raft_consensus.cc:359] T 3b4653bd3b624448801635ef21288354 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.828470 1730579 raft_consensus.cc:385] T 3b4653bd3b624448801635ef21288354 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.828686 1730579 raft_consensus.cc:740] T 3b4653bd3b624448801635ef21288354 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a727b6e247b14eb59af28df77d7afa22, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.829142 1730582 tablet_bootstrap.cc:492] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e: No bootstrap required, opened a new log
I20251027 02:47:38.829499 1730582 ts_tablet_manager.cc:1403] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20251027 02:47:38.829221 1730579 consensus_queue.cc:260] T 3b4653bd3b624448801635ef21288354 P a727b6e247b14eb59af28df77d7afa22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.830919 1730579 ts_tablet_manager.cc:1434] T 3b4653bd3b624448801635ef21288354 P a727b6e247b14eb59af28df77d7afa22: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20251027 02:47:38.831678 1730579 tablet_bootstrap.cc:492] T 1a5f4ca8542948ce9886a384b1f97b3a P a727b6e247b14eb59af28df77d7afa22: Bootstrap starting.
I20251027 02:47:38.831535 1730582 raft_consensus.cc:359] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.832064 1730582 raft_consensus.cc:385] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.832284 1730582 raft_consensus.cc:740] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5e7b8aa196948a8aab515b05ee9e95e, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.832747 1730582 consensus_queue.cc:260] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.833168 1730581 tablet_bootstrap.cc:492] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b: No bootstrap required, opened a new log
I20251027 02:47:38.833556 1730581 ts_tablet_manager.cc:1403] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.003s
I20251027 02:47:38.834336 1730582 ts_tablet_manager.cc:1434] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20251027 02:47:38.835327 1730582 tablet_bootstrap.cc:492] T 1a5f4ca8542948ce9886a384b1f97b3a P e5e7b8aa196948a8aab515b05ee9e95e: Bootstrap starting.
I20251027 02:47:38.835742 1730581 raft_consensus.cc:359] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.836349 1730581 raft_consensus.cc:385] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.836757 1730579 tablet_bootstrap.cc:654] T 1a5f4ca8542948ce9886a384b1f97b3a P a727b6e247b14eb59af28df77d7afa22: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.836645 1730581 raft_consensus.cc:740] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e48d138f3a134c448081b884d795716b, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.837488 1730581 consensus_queue.cc:260] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.839321 1730581 ts_tablet_manager.cc:1434] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20251027 02:47:38.839517 1730582 tablet_bootstrap.cc:654] T 1a5f4ca8542948ce9886a384b1f97b3a P e5e7b8aa196948a8aab515b05ee9e95e: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.840844 1730581 tablet_bootstrap.cc:492] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b: Bootstrap starting.
I20251027 02:47:38.843003 1730579 tablet_bootstrap.cc:492] T 1a5f4ca8542948ce9886a384b1f97b3a P a727b6e247b14eb59af28df77d7afa22: No bootstrap required, opened a new log
I20251027 02:47:38.843370 1730579 ts_tablet_manager.cc:1403] T 1a5f4ca8542948ce9886a384b1f97b3a P a727b6e247b14eb59af28df77d7afa22: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20251027 02:47:38.845579 1730581 tablet_bootstrap.cc:654] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.845695 1730582 tablet_bootstrap.cc:492] T 1a5f4ca8542948ce9886a384b1f97b3a P e5e7b8aa196948a8aab515b05ee9e95e: No bootstrap required, opened a new log
I20251027 02:47:38.846050 1730582 ts_tablet_manager.cc:1403] T 1a5f4ca8542948ce9886a384b1f97b3a P e5e7b8aa196948a8aab515b05ee9e95e: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20251027 02:47:38.846030 1730579 raft_consensus.cc:359] T 1a5f4ca8542948ce9886a384b1f97b3a P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.846577 1730579 raft_consensus.cc:385] T 1a5f4ca8542948ce9886a384b1f97b3a P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.846833 1730579 raft_consensus.cc:740] T 1a5f4ca8542948ce9886a384b1f97b3a P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a727b6e247b14eb59af28df77d7afa22, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.847383 1730579 consensus_queue.cc:260] T 1a5f4ca8542948ce9886a384b1f97b3a P a727b6e247b14eb59af28df77d7afa22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.847916 1730582 raft_consensus.cc:359] T 1a5f4ca8542948ce9886a384b1f97b3a P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.848467 1730582 raft_consensus.cc:385] T 1a5f4ca8542948ce9886a384b1f97b3a P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.848716 1730582 raft_consensus.cc:740] T 1a5f4ca8542948ce9886a384b1f97b3a P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5e7b8aa196948a8aab515b05ee9e95e, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.849171 1730579 ts_tablet_manager.cc:1434] T 1a5f4ca8542948ce9886a384b1f97b3a P a727b6e247b14eb59af28df77d7afa22: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20251027 02:47:38.849262 1730582 consensus_queue.cc:260] T 1a5f4ca8542948ce9886a384b1f97b3a P e5e7b8aa196948a8aab515b05ee9e95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.849915 1730579 tablet_bootstrap.cc:492] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22: Bootstrap starting.
I20251027 02:47:38.850955 1730582 ts_tablet_manager.cc:1434] T 1a5f4ca8542948ce9886a384b1f97b3a P e5e7b8aa196948a8aab515b05ee9e95e: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20251027 02:47:38.851776 1730582 tablet_bootstrap.cc:492] T 932afee710bd49159618bd88c0295333 P e5e7b8aa196948a8aab515b05ee9e95e: Bootstrap starting.
I20251027 02:47:38.852003 1730581 tablet_bootstrap.cc:492] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b: No bootstrap required, opened a new log
I20251027 02:47:38.852416 1730581 ts_tablet_manager.cc:1403] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20251027 02:47:38.854799 1730579 tablet_bootstrap.cc:654] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.854672 1730581 raft_consensus.cc:359] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.855324 1730581 raft_consensus.cc:385] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.855608 1730581 raft_consensus.cc:740] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e48d138f3a134c448081b884d795716b, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.856128 1730582 tablet_bootstrap.cc:654] T 932afee710bd49159618bd88c0295333 P e5e7b8aa196948a8aab515b05ee9e95e: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.856220 1730581 consensus_queue.cc:260] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.858355 1730581 ts_tablet_manager.cc:1434] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20251027 02:47:38.859418 1730581 tablet_bootstrap.cc:492] T ee405edc42594078b91898ef9fbf8b33 P e48d138f3a134c448081b884d795716b: Bootstrap starting.
I20251027 02:47:38.860481 1730579 tablet_bootstrap.cc:492] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22: No bootstrap required, opened a new log
I20251027 02:47:38.860859 1730579 ts_tablet_manager.cc:1403] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.008s
I20251027 02:47:38.862061 1730582 tablet_bootstrap.cc:492] T 932afee710bd49159618bd88c0295333 P e5e7b8aa196948a8aab515b05ee9e95e: No bootstrap required, opened a new log
I20251027 02:47:38.862449 1730582 ts_tablet_manager.cc:1403] T 932afee710bd49159618bd88c0295333 P e5e7b8aa196948a8aab515b05ee9e95e: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20251027 02:47:38.862905 1730579 raft_consensus.cc:359] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.863499 1730579 raft_consensus.cc:385] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.863761 1730579 raft_consensus.cc:740] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a727b6e247b14eb59af28df77d7afa22, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.864327 1730579 consensus_queue.cc:260] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.864393 1730582 raft_consensus.cc:359] T 932afee710bd49159618bd88c0295333 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.864889 1730582 raft_consensus.cc:385] T 932afee710bd49159618bd88c0295333 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.865198 1730582 raft_consensus.cc:740] T 932afee710bd49159618bd88c0295333 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5e7b8aa196948a8aab515b05ee9e95e, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.865738 1730582 consensus_queue.cc:260] T 932afee710bd49159618bd88c0295333 P e5e7b8aa196948a8aab515b05ee9e95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.866161 1730579 ts_tablet_manager.cc:1434] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20251027 02:47:38.866494 1730581 tablet_bootstrap.cc:654] T ee405edc42594078b91898ef9fbf8b33 P e48d138f3a134c448081b884d795716b: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.867448 1730582 ts_tablet_manager.cc:1434] T 932afee710bd49159618bd88c0295333 P e5e7b8aa196948a8aab515b05ee9e95e: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20251027 02:47:38.868629 1730582 tablet_bootstrap.cc:492] T ee405edc42594078b91898ef9fbf8b33 P e5e7b8aa196948a8aab515b05ee9e95e: Bootstrap starting.
I20251027 02:47:38.873145 1730581 tablet_bootstrap.cc:492] T ee405edc42594078b91898ef9fbf8b33 P e48d138f3a134c448081b884d795716b: No bootstrap required, opened a new log
I20251027 02:47:38.873555 1730581 ts_tablet_manager.cc:1403] T ee405edc42594078b91898ef9fbf8b33 P e48d138f3a134c448081b884d795716b: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20251027 02:47:38.873766 1730582 tablet_bootstrap.cc:654] T ee405edc42594078b91898ef9fbf8b33 P e5e7b8aa196948a8aab515b05ee9e95e: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.875826 1730581 raft_consensus.cc:359] T ee405edc42594078b91898ef9fbf8b33 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } }
I20251027 02:47:38.876470 1730581 raft_consensus.cc:385] T ee405edc42594078b91898ef9fbf8b33 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.876721 1730581 raft_consensus.cc:740] T ee405edc42594078b91898ef9fbf8b33 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e48d138f3a134c448081b884d795716b, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.877331 1730581 consensus_queue.cc:260] T ee405edc42594078b91898ef9fbf8b33 P e48d138f3a134c448081b884d795716b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } }
I20251027 02:47:38.879303 1730581 ts_tablet_manager.cc:1434] T ee405edc42594078b91898ef9fbf8b33 P e48d138f3a134c448081b884d795716b: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20251027 02:47:38.879797 1730582 tablet_bootstrap.cc:492] T ee405edc42594078b91898ef9fbf8b33 P e5e7b8aa196948a8aab515b05ee9e95e: No bootstrap required, opened a new log
I20251027 02:47:38.880095 1730581 tablet_bootstrap.cc:492] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b: Bootstrap starting.
I20251027 02:47:38.880141 1730582 ts_tablet_manager.cc:1403] T ee405edc42594078b91898ef9fbf8b33 P e5e7b8aa196948a8aab515b05ee9e95e: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20251027 02:47:38.882222 1730582 raft_consensus.cc:359] T ee405edc42594078b91898ef9fbf8b33 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } }
I20251027 02:47:38.882753 1730582 raft_consensus.cc:385] T ee405edc42594078b91898ef9fbf8b33 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.882987 1730582 raft_consensus.cc:740] T ee405edc42594078b91898ef9fbf8b33 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5e7b8aa196948a8aab515b05ee9e95e, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.883526 1730582 consensus_queue.cc:260] T ee405edc42594078b91898ef9fbf8b33 P e5e7b8aa196948a8aab515b05ee9e95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } }
I20251027 02:47:38.885382 1730582 ts_tablet_manager.cc:1434] T ee405edc42594078b91898ef9fbf8b33 P e5e7b8aa196948a8aab515b05ee9e95e: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20251027 02:47:38.886139 1730582 tablet_bootstrap.cc:492] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e: Bootstrap starting.
I20251027 02:47:38.886132 1730581 tablet_bootstrap.cc:654] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.891654 1730582 tablet_bootstrap.cc:654] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.891952 1730581 tablet_bootstrap.cc:492] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b: No bootstrap required, opened a new log
I20251027 02:47:38.892360 1730581 ts_tablet_manager.cc:1403] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20251027 02:47:38.894701 1730581 raft_consensus.cc:359] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.895309 1730581 raft_consensus.cc:385] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.895560 1730581 raft_consensus.cc:740] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e48d138f3a134c448081b884d795716b, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.896109 1730581 consensus_queue.cc:260] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.897409 1730582 tablet_bootstrap.cc:492] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e: No bootstrap required, opened a new log
I20251027 02:47:38.897776 1730582 ts_tablet_manager.cc:1403] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20251027 02:47:38.898075 1730581 ts_tablet_manager.cc:1434] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20251027 02:47:38.898880 1730581 tablet_bootstrap.cc:492] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e48d138f3a134c448081b884d795716b: Bootstrap starting.
I20251027 02:47:38.899822 1730582 raft_consensus.cc:359] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.900365 1730582 raft_consensus.cc:385] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.900595 1730582 raft_consensus.cc:740] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5e7b8aa196948a8aab515b05ee9e95e, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.901234 1730582 consensus_queue.cc:260] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.903033 1730582 ts_tablet_manager.cc:1434] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20251027 02:47:38.904860 1730581 tablet_bootstrap.cc:654] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e48d138f3a134c448081b884d795716b: Neither blocks nor log segments found. Creating new log.
I20251027 02:47:38.910779 1730581 tablet_bootstrap.cc:492] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e48d138f3a134c448081b884d795716b: No bootstrap required, opened a new log
I20251027 02:47:38.911182 1730581 ts_tablet_manager.cc:1403] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e48d138f3a134c448081b884d795716b: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20251027 02:47:38.913328 1730581 raft_consensus.cc:359] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.913921 1730581 raft_consensus.cc:385] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:47:38.914170 1730581 raft_consensus.cc:740] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e48d138f3a134c448081b884d795716b, State: Initialized, Role: FOLLOWER
I20251027 02:47:38.914752 1730581 consensus_queue.cc:260] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e48d138f3a134c448081b884d795716b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.916682 1730581 ts_tablet_manager.cc:1434] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e48d138f3a134c448081b884d795716b: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20251027 02:47:38.924775 1730200 tablet.cc:2378] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:47:38.945662 1730588 raft_consensus.cc:493] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:47:38.946203 1730588 raft_consensus.cc:515] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.949312 1730588 leader_election.cc:290] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e48d138f3a134c448081b884d795716b (127.152.231.193:44051), a727b6e247b14eb59af28df77d7afa22 (127.152.231.195:42803)
I20251027 02:47:38.965058 1730150 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b4653bd3b624448801635ef21288354" candidate_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48d138f3a134c448081b884d795716b" is_pre_election: true
I20251027 02:47:38.965884 1730150 raft_consensus.cc:2468] T 3b4653bd3b624448801635ef21288354 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5e7b8aa196948a8aab515b05ee9e95e in term 0.
I20251027 02:47:38.966508 1730494 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b4653bd3b624448801635ef21288354" candidate_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a727b6e247b14eb59af28df77d7afa22" is_pre_election: true
I20251027 02:47:38.967185 1730236 leader_election.cc:304] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e48d138f3a134c448081b884d795716b, e5e7b8aa196948a8aab515b05ee9e95e; no voters:
I20251027 02:47:38.967664 1730494 raft_consensus.cc:2468] T 3b4653bd3b624448801635ef21288354 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5e7b8aa196948a8aab515b05ee9e95e in term 0.
I20251027 02:47:38.968130 1730588 raft_consensus.cc:2804] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:47:38.968381 1730588 raft_consensus.cc:493] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:47:38.968578 1730588 raft_consensus.cc:3060] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:38.974642 1730588 raft_consensus.cc:515] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.975802 1730588 leader_election.cc:290] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [CANDIDATE]: Term 1 election: Requested vote from peers e48d138f3a134c448081b884d795716b (127.152.231.193:44051), a727b6e247b14eb59af28df77d7afa22 (127.152.231.195:42803)
I20251027 02:47:38.976779 1730150 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b4653bd3b624448801635ef21288354" candidate_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48d138f3a134c448081b884d795716b"
I20251027 02:47:38.976792 1730494 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "3b4653bd3b624448801635ef21288354" candidate_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a727b6e247b14eb59af28df77d7afa22"
I20251027 02:47:38.977200 1730150 raft_consensus.cc:3060] T 3b4653bd3b624448801635ef21288354 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:38.977224 1730494 raft_consensus.cc:3060] T 3b4653bd3b624448801635ef21288354 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:38.982414 1730494 raft_consensus.cc:2468] T 3b4653bd3b624448801635ef21288354 P a727b6e247b14eb59af28df77d7afa22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5e7b8aa196948a8aab515b05ee9e95e in term 1.
I20251027 02:47:38.982411 1730150 raft_consensus.cc:2468] T 3b4653bd3b624448801635ef21288354 P e48d138f3a134c448081b884d795716b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5e7b8aa196948a8aab515b05ee9e95e in term 1.
I20251027 02:47:38.983320 1730235 leader_election.cc:304] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a727b6e247b14eb59af28df77d7afa22, e5e7b8aa196948a8aab515b05ee9e95e; no voters:
I20251027 02:47:38.984035 1730588 raft_consensus.cc:2804] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:47:38.986025 1730588 raft_consensus.cc:697] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [term 1 LEADER]: Becoming Leader. State: Replica: e5e7b8aa196948a8aab515b05ee9e95e, State: Running, Role: LEADER
I20251027 02:47:38.986692 1730588 consensus_queue.cc:237] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:38.999925 1729933 catalog_manager.cc:5649] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e reported cstate change: term changed from 0 to 1, leader changed from <none> to e5e7b8aa196948a8aab515b05ee9e95e (127.152.231.194). New cstate: current_term: 1 leader_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } health_report { overall_health: UNKNOWN } } }
I20251027 02:47:39.006999 1730589 raft_consensus.cc:493] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:47:39.007985 1730589 raft_consensus.cc:515] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:39.011585 1730589 leader_election.cc:290] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5e7b8aa196948a8aab515b05ee9e95e (127.152.231.194:35939), a727b6e247b14eb59af28df77d7afa22 (127.152.231.195:42803)
W20251027 02:47:39.017858 1730352 tablet.cc:2378] T 028cac69bd2c4adda9da6b2e07fcee84 P e5e7b8aa196948a8aab515b05ee9e95e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:47:39.025207 1730583 raft_consensus.cc:493] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251027 02:47:39.025398 1730549 tablet.cc:2378] T 028cac69bd2c4adda9da6b2e07fcee84 P a727b6e247b14eb59af28df77d7afa22: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:47:39.025684 1730583 raft_consensus.cc:515] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } }
I20251027 02:47:39.027674 1730494 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a5f4ca8542948ce9886a384b1f97b3a" candidate_uuid: "e48d138f3a134c448081b884d795716b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a727b6e247b14eb59af28df77d7afa22" is_pre_election: true
I20251027 02:47:39.028247 1730494 raft_consensus.cc:2468] T 1a5f4ca8542948ce9886a384b1f97b3a P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e48d138f3a134c448081b884d795716b in term 0.
I20251027 02:47:39.029040 1730583 leader_election.cc:290] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5e7b8aa196948a8aab515b05ee9e95e (127.152.231.194:35939), e48d138f3a134c448081b884d795716b (127.152.231.193:44051)
I20251027 02:47:39.029884 1730083 leader_election.cc:304] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a727b6e247b14eb59af28df77d7afa22, e48d138f3a134c448081b884d795716b; no voters:
I20251027 02:47:39.030706 1730589 raft_consensus.cc:2804] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:47:39.030944 1730589 raft_consensus.cc:493] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:47:39.031147 1730589 raft_consensus.cc:3060] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:39.030948 1730302 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a5f4ca8542948ce9886a384b1f97b3a" candidate_uuid: "e48d138f3a134c448081b884d795716b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" is_pre_election: true
I20251027 02:47:39.031796 1730302 raft_consensus.cc:2468] T 1a5f4ca8542948ce9886a384b1f97b3a P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e48d138f3a134c448081b884d795716b in term 0.
I20251027 02:47:39.035795 1730589 raft_consensus.cc:515] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:39.036433 1730611 raft_consensus.cc:493] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:47:39.036765 1730611 raft_consensus.cc:515] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:39.037129 1730589 leader_election.cc:290] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [CANDIDATE]: Term 1 election: Requested vote from peers e5e7b8aa196948a8aab515b05ee9e95e (127.152.231.194:35939), a727b6e247b14eb59af28df77d7afa22 (127.152.231.195:42803)
I20251027 02:47:39.037626 1730150 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee405edc42594078b91898ef9fbf8b33" candidate_uuid: "a727b6e247b14eb59af28df77d7afa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48d138f3a134c448081b884d795716b" is_pre_election: true
I20251027 02:47:39.037869 1730302 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a5f4ca8542948ce9886a384b1f97b3a" candidate_uuid: "e48d138f3a134c448081b884d795716b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5e7b8aa196948a8aab515b05ee9e95e"
I20251027 02:47:39.038028 1730494 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1a5f4ca8542948ce9886a384b1f97b3a" candidate_uuid: "e48d138f3a134c448081b884d795716b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a727b6e247b14eb59af28df77d7afa22"
I20251027 02:47:39.038154 1730150 raft_consensus.cc:2468] T ee405edc42594078b91898ef9fbf8b33 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a727b6e247b14eb59af28df77d7afa22 in term 0.
I20251027 02:47:39.038384 1730494 raft_consensus.cc:3060] T 1a5f4ca8542948ce9886a384b1f97b3a P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:39.038443 1730302 raft_consensus.cc:3060] T 1a5f4ca8542948ce9886a384b1f97b3a P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:39.038362 1730301 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee405edc42594078b91898ef9fbf8b33" candidate_uuid: "a727b6e247b14eb59af28df77d7afa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" is_pre_election: true
I20251027 02:47:39.039103 1730611 leader_election.cc:290] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5e7b8aa196948a8aab515b05ee9e95e (127.152.231.194:35939), a727b6e247b14eb59af28df77d7afa22 (127.152.231.195:42803)
I20251027 02:47:39.039031 1730301 raft_consensus.cc:2468] T ee405edc42594078b91898ef9fbf8b33 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a727b6e247b14eb59af28df77d7afa22 in term 0.
I20251027 02:47:39.039142 1730424 leader_election.cc:304] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a727b6e247b14eb59af28df77d7afa22, e48d138f3a134c448081b884d795716b; no voters:
I20251027 02:47:39.040323 1730300 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "932afee710bd49159618bd88c0295333" candidate_uuid: "e48d138f3a134c448081b884d795716b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" is_pre_election: true
I20251027 02:47:39.040701 1730583 raft_consensus.cc:2804] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:47:39.041006 1730583 raft_consensus.cc:493] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:47:39.041026 1730300 raft_consensus.cc:2468] T 932afee710bd49159618bd88c0295333 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e48d138f3a134c448081b884d795716b in term 0.
I20251027 02:47:39.041070 1730493 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "932afee710bd49159618bd88c0295333" candidate_uuid: "e48d138f3a134c448081b884d795716b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a727b6e247b14eb59af28df77d7afa22" is_pre_election: true
I20251027 02:47:39.041379 1730583 raft_consensus.cc:3060] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:39.041932 1730493 raft_consensus.cc:2468] T 932afee710bd49159618bd88c0295333 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e48d138f3a134c448081b884d795716b in term 0.
I20251027 02:47:39.042191 1730086 leader_election.cc:304] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e48d138f3a134c448081b884d795716b, e5e7b8aa196948a8aab515b05ee9e95e; no voters:
I20251027 02:47:39.043041 1730611 raft_consensus.cc:2804] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:47:39.043270 1730611 raft_consensus.cc:493] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:47:39.043283 1730494 raft_consensus.cc:2468] T 1a5f4ca8542948ce9886a384b1f97b3a P a727b6e247b14eb59af28df77d7afa22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e48d138f3a134c448081b884d795716b in term 1.
I20251027 02:47:39.043469 1730611 raft_consensus.cc:3060] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:39.044091 1730083 leader_election.cc:304] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a727b6e247b14eb59af28df77d7afa22, e48d138f3a134c448081b884d795716b; no voters:
I20251027 02:47:39.044692 1730589 raft_consensus.cc:2804] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:47:39.045830 1730302 raft_consensus.cc:2468] T 1a5f4ca8542948ce9886a384b1f97b3a P e5e7b8aa196948a8aab515b05ee9e95e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e48d138f3a134c448081b884d795716b in term 1.
I20251027 02:47:39.046872 1730589 raft_consensus.cc:697] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [term 1 LEADER]: Becoming Leader. State: Replica: e48d138f3a134c448081b884d795716b, State: Running, Role: LEADER
I20251027 02:47:39.047505 1730589 consensus_queue.cc:237] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:39.048445 1730583 raft_consensus.cc:515] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } }
I20251027 02:47:39.048522 1730611 raft_consensus.cc:515] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:39.048905 1730612 raft_consensus.cc:493] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:47:39.049360 1730612 raft_consensus.cc:515] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:39.049916 1730583 leader_election.cc:290] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [CANDIDATE]: Term 1 election: Requested vote from peers e5e7b8aa196948a8aab515b05ee9e95e (127.152.231.194:35939), e48d138f3a134c448081b884d795716b (127.152.231.193:44051)
I20251027 02:47:39.050415 1730611 leader_election.cc:290] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [CANDIDATE]: Term 1 election: Requested vote from peers e5e7b8aa196948a8aab515b05ee9e95e (127.152.231.194:35939), a727b6e247b14eb59af28df77d7afa22 (127.152.231.195:42803)
I20251027 02:47:39.050668 1730302 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee405edc42594078b91898ef9fbf8b33" candidate_uuid: "a727b6e247b14eb59af28df77d7afa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5e7b8aa196948a8aab515b05ee9e95e"
I20251027 02:47:39.050923 1730150 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ee405edc42594078b91898ef9fbf8b33" candidate_uuid: "a727b6e247b14eb59af28df77d7afa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48d138f3a134c448081b884d795716b"
I20251027 02:47:39.050940 1730300 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "932afee710bd49159618bd88c0295333" candidate_uuid: "e48d138f3a134c448081b884d795716b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5e7b8aa196948a8aab515b05ee9e95e"
I20251027 02:47:39.051154 1730302 raft_consensus.cc:3060] T ee405edc42594078b91898ef9fbf8b33 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:39.051375 1730150 raft_consensus.cc:3060] T ee405edc42594078b91898ef9fbf8b33 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:39.051352 1730494 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "932afee710bd49159618bd88c0295333" candidate_uuid: "e48d138f3a134c448081b884d795716b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a727b6e247b14eb59af28df77d7afa22"
I20251027 02:47:39.051469 1730300 raft_consensus.cc:3060] T 932afee710bd49159618bd88c0295333 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:39.051787 1730494 raft_consensus.cc:3060] T 932afee710bd49159618bd88c0295333 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:39.052186 1730493 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "028cac69bd2c4adda9da6b2e07fcee84" candidate_uuid: "e48d138f3a134c448081b884d795716b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a727b6e247b14eb59af28df77d7afa22" is_pre_election: true
I20251027 02:47:39.051262 1730612 leader_election.cc:290] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5e7b8aa196948a8aab515b05ee9e95e (127.152.231.194:35939), a727b6e247b14eb59af28df77d7afa22 (127.152.231.195:42803)
I20251027 02:47:39.052526 1730301 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "028cac69bd2c4adda9da6b2e07fcee84" candidate_uuid: "e48d138f3a134c448081b884d795716b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" is_pre_election: true
I20251027 02:47:39.052733 1730493 raft_consensus.cc:2468] T 028cac69bd2c4adda9da6b2e07fcee84 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e48d138f3a134c448081b884d795716b in term 0.
I20251027 02:47:39.053048 1730301 raft_consensus.cc:2468] T 028cac69bd2c4adda9da6b2e07fcee84 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e48d138f3a134c448081b884d795716b in term 0.
I20251027 02:47:39.053649 1730083 leader_election.cc:304] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a727b6e247b14eb59af28df77d7afa22, e48d138f3a134c448081b884d795716b; no voters:
I20251027 02:47:39.054349 1730612 raft_consensus.cc:2804] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:47:39.054675 1730612 raft_consensus.cc:493] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:47:39.054979 1730612 raft_consensus.cc:3060] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:39.056180 1730494 raft_consensus.cc:2468] T 932afee710bd49159618bd88c0295333 P a727b6e247b14eb59af28df77d7afa22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e48d138f3a134c448081b884d795716b in term 1.
I20251027 02:47:39.056346 1729933 catalog_manager.cc:5649] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b reported cstate change: term changed from 0 to 1, leader changed from <none> to e48d138f3a134c448081b884d795716b (127.152.231.193). New cstate: current_term: 1 leader_uuid: "e48d138f3a134c448081b884d795716b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } health_report { overall_health: UNKNOWN } } }
I20251027 02:47:39.056672 1730302 raft_consensus.cc:2468] T ee405edc42594078b91898ef9fbf8b33 P e5e7b8aa196948a8aab515b05ee9e95e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a727b6e247b14eb59af28df77d7afa22 in term 1.
I20251027 02:47:39.056977 1730083 leader_election.cc:304] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a727b6e247b14eb59af28df77d7afa22, e48d138f3a134c448081b884d795716b; no voters:
I20251027 02:47:39.057608 1730589 raft_consensus.cc:2804] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:47:39.057654 1730426 leader_election.cc:304] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a727b6e247b14eb59af28df77d7afa22, e5e7b8aa196948a8aab515b05ee9e95e; no voters:
I20251027 02:47:39.057595 1730150 raft_consensus.cc:2468] T ee405edc42594078b91898ef9fbf8b33 P e48d138f3a134c448081b884d795716b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a727b6e247b14eb59af28df77d7afa22 in term 1.
I20251027 02:47:39.058005 1730589 raft_consensus.cc:697] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [term 1 LEADER]: Becoming Leader. State: Replica: e48d138f3a134c448081b884d795716b, State: Running, Role: LEADER
I20251027 02:47:39.058349 1730583 raft_consensus.cc:2804] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:47:39.058555 1730589 consensus_queue.cc:237] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:39.058988 1730300 raft_consensus.cc:2468] T 932afee710bd49159618bd88c0295333 P e5e7b8aa196948a8aab515b05ee9e95e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e48d138f3a134c448081b884d795716b in term 1.
I20251027 02:47:39.059818 1730612 raft_consensus.cc:515] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:39.060648 1730583 raft_consensus.cc:697] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [term 1 LEADER]: Becoming Leader. State: Replica: a727b6e247b14eb59af28df77d7afa22, State: Running, Role: LEADER
I20251027 02:47:39.060878 1730612 leader_election.cc:290] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [CANDIDATE]: Term 1 election: Requested vote from peers e5e7b8aa196948a8aab515b05ee9e95e (127.152.231.194:35939), a727b6e247b14eb59af28df77d7afa22 (127.152.231.195:42803)
I20251027 02:47:39.061532 1730583 consensus_queue.cc:237] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } }
I20251027 02:47:39.061895 1730494 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "028cac69bd2c4adda9da6b2e07fcee84" candidate_uuid: "e48d138f3a134c448081b884d795716b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a727b6e247b14eb59af28df77d7afa22"
I20251027 02:47:39.062251 1730300 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "028cac69bd2c4adda9da6b2e07fcee84" candidate_uuid: "e48d138f3a134c448081b884d795716b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5e7b8aa196948a8aab515b05ee9e95e"
I20251027 02:47:39.062472 1730494 raft_consensus.cc:3060] T 028cac69bd2c4adda9da6b2e07fcee84 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:39.062703 1730300 raft_consensus.cc:3060] T 028cac69bd2c4adda9da6b2e07fcee84 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:39.067902 1730300 raft_consensus.cc:2468] T 028cac69bd2c4adda9da6b2e07fcee84 P e5e7b8aa196948a8aab515b05ee9e95e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e48d138f3a134c448081b884d795716b in term 1.
I20251027 02:47:39.068118 1730494 raft_consensus.cc:2468] T 028cac69bd2c4adda9da6b2e07fcee84 P a727b6e247b14eb59af28df77d7afa22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e48d138f3a134c448081b884d795716b in term 1.
I20251027 02:47:39.068759 1730086 leader_election.cc:304] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e48d138f3a134c448081b884d795716b, e5e7b8aa196948a8aab515b05ee9e95e; no voters:
I20251027 02:47:39.069475 1730589 raft_consensus.cc:2804] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:47:39.069780 1730589 raft_consensus.cc:697] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [term 1 LEADER]: Becoming Leader. State: Replica: e48d138f3a134c448081b884d795716b, State: Running, Role: LEADER
I20251027 02:47:39.070360 1730589 consensus_queue.cc:237] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:39.072206 1729934 catalog_manager.cc:5649] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 reported cstate change: term changed from 0 to 1, leader changed from <none> to a727b6e247b14eb59af28df77d7afa22 (127.152.231.195). New cstate: current_term: 1 leader_uuid: "a727b6e247b14eb59af28df77d7afa22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } health_report { overall_health: UNKNOWN } } }
I20251027 02:47:39.072088 1729933 catalog_manager.cc:5649] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b reported cstate change: term changed from 0 to 1, leader changed from <none> to e48d138f3a134c448081b884d795716b (127.152.231.193). New cstate: current_term: 1 leader_uuid: "e48d138f3a134c448081b884d795716b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } health_report { overall_health: UNKNOWN } } }
I20251027 02:47:39.092207 1729933 catalog_manager.cc:5649] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b reported cstate change: term changed from 0 to 1, leader changed from <none> to e48d138f3a134c448081b884d795716b (127.152.231.193). New cstate: current_term: 1 leader_uuid: "e48d138f3a134c448081b884d795716b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } health_report { overall_health: UNKNOWN } } }
I20251027 02:47:39.181771 1730583 raft_consensus.cc:493] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:47:39.182215 1730583 raft_consensus.cc:515] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:39.183871 1730583 leader_election.cc:290] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e48d138f3a134c448081b884d795716b (127.152.231.193:44051), e5e7b8aa196948a8aab515b05ee9e95e (127.152.231.194:35939)
I20251027 02:47:39.184443 1730588 raft_consensus.cc:493] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:47:39.184667 1730150 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "56a1f0cb4ab34f2fb12c1fadfea103b7" candidate_uuid: "a727b6e247b14eb59af28df77d7afa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48d138f3a134c448081b884d795716b" is_pre_election: true
I20251027 02:47:39.184762 1730300 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "56a1f0cb4ab34f2fb12c1fadfea103b7" candidate_uuid: "a727b6e247b14eb59af28df77d7afa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" is_pre_election: true
I20251027 02:47:39.184839 1730588 raft_consensus.cc:515] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:39.185279 1730150 raft_consensus.cc:2468] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a727b6e247b14eb59af28df77d7afa22 in term 0.
I20251027 02:47:39.185761 1730300 raft_consensus.cc:2468] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a727b6e247b14eb59af28df77d7afa22 in term 0.
I20251027 02:47:39.186159 1730424 leader_election.cc:304] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a727b6e247b14eb59af28df77d7afa22, e48d138f3a134c448081b884d795716b; no voters:
I20251027 02:47:39.186453 1730588 leader_election.cc:290] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e48d138f3a134c448081b884d795716b (127.152.231.193:44051), a727b6e247b14eb59af28df77d7afa22 (127.152.231.195:42803)
I20251027 02:47:39.186772 1730583 raft_consensus.cc:2804] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:47:39.187043 1730583 raft_consensus.cc:493] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:47:39.187065 1730150 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "56a1f0cb4ab34f2fb12c1fadfea103b7" candidate_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48d138f3a134c448081b884d795716b" is_pre_election: true
I20251027 02:47:39.187198 1730494 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "56a1f0cb4ab34f2fb12c1fadfea103b7" candidate_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a727b6e247b14eb59af28df77d7afa22" is_pre_election: true
I20251027 02:47:39.187286 1730583 raft_consensus.cc:3060] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:39.187568 1730150 raft_consensus.cc:2468] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5e7b8aa196948a8aab515b05ee9e95e in term 0.
I20251027 02:47:39.188297 1730236 leader_election.cc:304] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e48d138f3a134c448081b884d795716b, e5e7b8aa196948a8aab515b05ee9e95e; no voters:
I20251027 02:47:39.188794 1730588 raft_consensus.cc:2804] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:47:39.189101 1730588 raft_consensus.cc:493] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:47:39.189373 1730588 raft_consensus.cc:3060] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:39.193301 1730583 raft_consensus.cc:515] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:39.194239 1730494 raft_consensus.cc:2393] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e5e7b8aa196948a8aab515b05ee9e95e in current term 1: Already voted for candidate a727b6e247b14eb59af28df77d7afa22 in this term.
I20251027 02:47:39.194801 1730583 leader_election.cc:290] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [CANDIDATE]: Term 1 election: Requested vote from peers e48d138f3a134c448081b884d795716b (127.152.231.193:44051), e5e7b8aa196948a8aab515b05ee9e95e (127.152.231.194:35939)
I20251027 02:47:39.195340 1730588 raft_consensus.cc:515] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:39.195611 1730150 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "56a1f0cb4ab34f2fb12c1fadfea103b7" candidate_uuid: "a727b6e247b14eb59af28df77d7afa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48d138f3a134c448081b884d795716b"
I20251027 02:47:39.196087 1730150 raft_consensus.cc:3060] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e48d138f3a134c448081b884d795716b [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:47:39.196213 1730300 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "56a1f0cb4ab34f2fb12c1fadfea103b7" candidate_uuid: "a727b6e247b14eb59af28df77d7afa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5e7b8aa196948a8aab515b05ee9e95e"
I20251027 02:47:39.196875 1730300 raft_consensus.cc:2393] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a727b6e247b14eb59af28df77d7afa22 in current term 1: Already voted for candidate e5e7b8aa196948a8aab515b05ee9e95e in this term.
I20251027 02:47:39.196846 1730588 leader_election.cc:290] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e [CANDIDATE]: Term 1 election: Requested vote from peers e48d138f3a134c448081b884d795716b (127.152.231.193:44051), a727b6e247b14eb59af28df77d7afa22 (127.152.231.195:42803)
I20251027 02:47:39.197860 1730494 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "56a1f0cb4ab34f2fb12c1fadfea103b7" candidate_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a727b6e247b14eb59af28df77d7afa22"
I20251027 02:47:39.198290 1730494 raft_consensus.cc:2393] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e5e7b8aa196948a8aab515b05ee9e95e in current term 1: Already voted for candidate a727b6e247b14eb59af28df77d7afa22 in this term.
I20251027 02:47:39.197772 1730149 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "56a1f0cb4ab34f2fb12c1fadfea103b7" candidate_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48d138f3a134c448081b884d795716b"
I20251027 02:47:39.201182 1730236 leader_election.cc:304] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e5e7b8aa196948a8aab515b05ee9e95e; no voters: a727b6e247b14eb59af28df77d7afa22, e48d138f3a134c448081b884d795716b
I20251027 02:47:39.201731 1730588 raft_consensus.cc:2749] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e5e7b8aa196948a8aab515b05ee9e95e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20251027 02:47:39.203078 1730150 raft_consensus.cc:2468] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P e48d138f3a134c448081b884d795716b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a727b6e247b14eb59af28df77d7afa22 in term 1.
I20251027 02:47:39.203948 1730424 leader_election.cc:304] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a727b6e247b14eb59af28df77d7afa22, e48d138f3a134c448081b884d795716b; no voters: e5e7b8aa196948a8aab515b05ee9e95e
I20251027 02:47:39.204514 1730583 raft_consensus.cc:2804] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:47:39.204895 1730583 raft_consensus.cc:697] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [term 1 LEADER]: Becoming Leader. State: Replica: a727b6e247b14eb59af28df77d7afa22, State: Running, Role: LEADER
I20251027 02:47:39.205530 1730583 consensus_queue.cc:237] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } }
I20251027 02:47:39.214183 1729933 catalog_manager.cc:5649] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 reported cstate change: term changed from 0 to 1, leader changed from <none> to a727b6e247b14eb59af28df77d7afa22 (127.152.231.195). New cstate: current_term: 1 leader_uuid: "a727b6e247b14eb59af28df77d7afa22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 } health_report { overall_health: HEALTHY } } }
I20251027 02:47:39.441640 1730589 consensus_queue.cc:1048] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:47:39.455322 1730612 consensus_queue.cc:1048] T 932afee710bd49159618bd88c0295333 P e48d138f3a134c448081b884d795716b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:47:39.460429 1730612 consensus_queue.cc:1048] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:47:39.471221 1730612 consensus_queue.cc:1048] T 028cac69bd2c4adda9da6b2e07fcee84 P e48d138f3a134c448081b884d795716b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:47:39.509331 1730588 consensus_queue.cc:1048] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:47:39.521281 1730643 consensus_queue.cc:1048] T 3b4653bd3b624448801635ef21288354 P e5e7b8aa196948a8aab515b05ee9e95e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:47:39.533313 1730612 consensus_queue.cc:1048] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:47:39.543910 1730589 consensus_queue.cc:1048] T 1a5f4ca8542948ce9886a384b1f97b3a P e48d138f3a134c448081b884d795716b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a727b6e247b14eb59af28df77d7afa22" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 42803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:47:39.611222 1730583 consensus_queue.cc:1048] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:47:39.621661 1730614 consensus_queue.cc:1048] T ee405edc42594078b91898ef9fbf8b33 P a727b6e247b14eb59af28df77d7afa22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:47:39.778409 1730614 consensus_queue.cc:1048] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5e7b8aa196948a8aab515b05ee9e95e" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:47:39.788655 1730583 consensus_queue.cc:1048] T 56a1f0cb4ab34f2fb12c1fadfea103b7 P a727b6e247b14eb59af28df77d7afa22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48d138f3a134c448081b884d795716b" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 44051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251027 02:47:43.236037 1730733 debug-util.cc:398] Leaking SignalData structure 0x7b08000b3f40 after lost signal to thread 1729486
W20251027 02:47:43.236578 1730733 debug-util.cc:398] Leaking SignalData structure 0x7b0800050e00 after lost signal to thread 1729562
W20251027 02:47:43.475355 1729965 debug-util.cc:398] Leaking SignalData structure 0x7b08000abc00 after lost signal to thread 1729892
W20251027 02:47:43.475909 1729965 debug-util.cc:398] Leaking SignalData structure 0x7b080003cf80 after lost signal to thread 1729968
W20251027 02:47:49.836148 1730906 debug-util.cc:398] Leaking SignalData structure 0x7b080009c860 after lost signal to thread 1729569
W20251027 02:47:49.836876 1730906 debug-util.cc:398] Leaking SignalData structure 0x7b08000a7da0 after lost signal to thread 1729884
W20251027 02:47:57.068478 1729965 debug-util.cc:398] Leaking SignalData structure 0x7b080009e9a0 after lost signal to thread 1729892
W20251027 02:47:57.069226 1729965 debug-util.cc:398] Leaking SignalData structure 0x7b0800073bc0 after lost signal to thread 1729968
W20251027 02:47:58.268096 1731196 debug-util.cc:398] Leaking SignalData structure 0x7b08000ae860 after lost signal to thread 1729892
W20251027 02:47:58.278589 1731196 debug-util.cc:398] Leaking SignalData structure 0x7b0800099860 after lost signal to thread 1729968
W20251027 02:48:07.969461 1731244 debug-util.cc:398] Leaking SignalData structure 0x7b08000dbac0 after lost signal to thread 1730013
W20251027 02:48:07.970733 1731244 debug-util.cc:398] Leaking SignalData structure 0x7b08000cf5e0 after lost signal to thread 1730195
W20251027 02:48:18.556680 1731475 debug-util.cc:398] Leaking SignalData structure 0x7b08000e8de0 after lost signal to thread 1730208
W20251027 02:48:18.558053 1731475 debug-util.cc:398] Leaking SignalData structure 0x7b080005a260 after lost signal to thread 1730346
W20251027 02:48:20.074692 1729559 debug-util.cc:398] Leaking SignalData structure 0x7b0800073a60 after lost signal to thread 1729486
W20251027 02:48:20.075481 1729559 debug-util.cc:398] Leaking SignalData structure 0x7b08000b3460 after lost signal to thread 1729562
W20251027 02:48:29.232705 1731721 debug-util.cc:398] Leaking SignalData structure 0x7b08000b52c0 after lost signal to thread 1730360
W20251027 02:48:29.233963 1731721 debug-util.cc:398] Leaking SignalData structure 0x7b08000810a0 after lost signal to thread 1730540
W20251027 02:48:32.549669 1730343 debug-util.cc:398] Leaking SignalData structure 0x7b08000e4a20 after lost signal to thread 1730208
W20251027 02:48:32.550941 1730343 debug-util.cc:398] Leaking SignalData structure 0x7b08000f3ac0 after lost signal to thread 1730346
I20251027 02:48:37.793608 1731244 scanners.cc:360] slow scan reporting is disabled: set --show_slow_scans to enable
I20251027 02:48:39.236124 1731475 scanners.cc:360] slow scan reporting is disabled: set --show_slow_scans to enable
W20251027 02:48:40.509786 1729881 debug-util.cc:398] Leaking SignalData structure 0x7b08000a2fc0 after lost signal to thread 1729569
W20251027 02:48:40.510511 1729881 debug-util.cc:398] Leaking SignalData structure 0x7b080009ca40 after lost signal to thread 1729884
I20251027 02:48:40.581486 1731721 scanners.cc:360] slow scan reporting is disabled: set --show_slow_scans to enable
I20251027 02:48:43.351749 1729439 webserver-crawl-itest.cc:326] Dumping visited URLs
I20251027 02:48:43.352130 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599
I20251027 02:48:43.352308 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/bootstrap/js/bootstrap-table.min.js
I20251027 02:48:43.352463 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/bootstrap/js/bootstrap.min.js
I20251027 02:48:43.352612 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/config
I20251027 02:48:43.352758 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/dashboards
I20251027 02:48:43.352905 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/healthz
I20251027 02:48:43.353048 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/jquery-3.5.1.min.js
I20251027 02:48:43.353226 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/kudu.js
I20251027 02:48:43.353374 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/log-anchors?id=028cac69bd2c4adda9da6b2e07fcee84
I20251027 02:48:43.353518 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/log-anchors?id=1a5f4ca8542948ce9886a384b1f97b3a
I20251027 02:48:43.353657 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/log-anchors?id=3b4653bd3b624448801635ef21288354
I20251027 02:48:43.353796 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/log-anchors?id=56a1f0cb4ab34f2fb12c1fadfea103b7
I20251027 02:48:43.353930 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/log-anchors?id=932afee710bd49159618bd88c0295333
I20251027 02:48:43.354069 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/log-anchors?id=ee405edc42594078b91898ef9fbf8b33
I20251027 02:48:43.354216 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/logs
I20251027 02:48:43.354354 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/maintenance-manager
I20251027 02:48:43.354496 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/mem-trackers
I20251027 02:48:43.354638 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/memz
I20251027 02:48:43.354781 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/metrics
I20251027 02:48:43.354924 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/metrics_prometheus
I20251027 02:48:43.355062 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/rpcz
I20251027 02:48:43.355199 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/scans
I20251027 02:48:43.355363 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/stacks
I20251027 02:48:43.355506 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/startup
I20251027 02:48:43.355646 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet-consensus-status?id=028cac69bd2c4adda9da6b2e07fcee84
I20251027 02:48:43.355784 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet-consensus-status?id=1a5f4ca8542948ce9886a384b1f97b3a
I20251027 02:48:43.355931 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet-consensus-status?id=3b4653bd3b624448801635ef21288354
I20251027 02:48:43.356072 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet-consensus-status?id=56a1f0cb4ab34f2fb12c1fadfea103b7
I20251027 02:48:43.356215 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet-consensus-status?id=932afee710bd49159618bd88c0295333
I20251027 02:48:43.356367 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet-consensus-status?id=ee405edc42594078b91898ef9fbf8b33
I20251027 02:48:43.356508 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet-rowsetlayout-svg?id=028cac69bd2c4adda9da6b2e07fcee84
I20251027 02:48:43.356649 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet-rowsetlayout-svg?id=1a5f4ca8542948ce9886a384b1f97b3a
I20251027 02:48:43.356793 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet-rowsetlayout-svg?id=3b4653bd3b624448801635ef21288354
I20251027 02:48:43.356937 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet-rowsetlayout-svg?id=56a1f0cb4ab34f2fb12c1fadfea103b7
I20251027 02:48:43.357097 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet-rowsetlayout-svg?id=932afee710bd49159618bd88c0295333
I20251027 02:48:43.357249 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet-rowsetlayout-svg?id=ee405edc42594078b91898ef9fbf8b33
I20251027 02:48:43.357378 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet?id=028cac69bd2c4adda9da6b2e07fcee84
I20251027 02:48:43.357512 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet?id=1a5f4ca8542948ce9886a384b1f97b3a
I20251027 02:48:43.357645 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet?id=3b4653bd3b624448801635ef21288354
I20251027 02:48:43.357803 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet?id=56a1f0cb4ab34f2fb12c1fadfea103b7
I20251027 02:48:43.357942 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet?id=932afee710bd49159618bd88c0295333
I20251027 02:48:43.358085 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablet?id=ee405edc42594078b91898ef9fbf8b33
I20251027 02:48:43.358290 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/tablets
I20251027 02:48:43.358486 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz
I20251027 02:48:43.358641 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=acceptor%20pool
I20251027 02:48:43.358790 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=all
I20251027 02:48:43.358948 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=cache
I20251027 02:48:43.359094 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=cloud%20detector
I20251027 02:48:43.359243 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=heartbeater
I20251027 02:48:43.359386 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=kernel-watchdog
I20251027 02:48:43.359527 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=maintenance
I20251027 02:48:43.359668 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=ntp
I20251027 02:48:43.359808 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=reactor
I20251027 02:48:43.359946 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=scanners
I20251027 02:48:43.360107 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=server
I20251027 02:48:43.360256 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:48:43.360399 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:48:43.360540 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:48:43.360680 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=service%20pool%20kudu.tserver.TabletServerAdminService
I20251027 02:48:43.360824 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=service%20pool%20kudu.tserver.TabletServerService
I20251027 02:48:43.360967 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=tablet-copy
I20251027 02:48:43.361133 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/threadz?group=thread%20pool
I20251027 02:48:43.361286 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/transactions
I20251027 02:48:43.361428 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:42599/varz
I20251027 02:48:43.361585 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311
I20251027 02:48:43.361733 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/bootstrap/js/bootstrap-table.min.js
I20251027 02:48:43.361879 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/bootstrap/js/bootstrap.min.js
I20251027 02:48:43.362023 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/config
I20251027 02:48:43.362167 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/dashboards
I20251027 02:48:43.362314 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/healthz
I20251027 02:48:43.362454 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/jquery-3.5.1.min.js
I20251027 02:48:43.362596 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/kudu.js
I20251027 02:48:43.362735 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/log-anchors?id=028cac69bd2c4adda9da6b2e07fcee84
I20251027 02:48:43.362874 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/log-anchors?id=1a5f4ca8542948ce9886a384b1f97b3a
I20251027 02:48:43.363029 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/log-anchors?id=3b4653bd3b624448801635ef21288354
I20251027 02:48:43.363169 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/log-anchors?id=56a1f0cb4ab34f2fb12c1fadfea103b7
I20251027 02:48:43.363340 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/log-anchors?id=932afee710bd49159618bd88c0295333
I20251027 02:48:43.363485 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/log-anchors?id=ee405edc42594078b91898ef9fbf8b33
I20251027 02:48:43.363629 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/logs
I20251027 02:48:43.363771 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/maintenance-manager
I20251027 02:48:43.363926 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/mem-trackers
I20251027 02:48:43.364082 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/memz
I20251027 02:48:43.364249 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/metrics
I20251027 02:48:43.364408 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/metrics_prometheus
I20251027 02:48:43.364562 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/rpcz
I20251027 02:48:43.364720 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/scans
I20251027 02:48:43.364872 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/stacks
I20251027 02:48:43.365031 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/startup
I20251027 02:48:43.365202 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet-consensus-status?id=028cac69bd2c4adda9da6b2e07fcee84
I20251027 02:48:43.365361 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet-consensus-status?id=1a5f4ca8542948ce9886a384b1f97b3a
I20251027 02:48:43.365505 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet-consensus-status?id=3b4653bd3b624448801635ef21288354
I20251027 02:48:43.365653 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet-consensus-status?id=56a1f0cb4ab34f2fb12c1fadfea103b7
I20251027 02:48:43.365801 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet-consensus-status?id=932afee710bd49159618bd88c0295333
I20251027 02:48:43.365950 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet-consensus-status?id=ee405edc42594078b91898ef9fbf8b33
I20251027 02:48:43.366109 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet-rowsetlayout-svg?id=028cac69bd2c4adda9da6b2e07fcee84
I20251027 02:48:43.366276 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet-rowsetlayout-svg?id=1a5f4ca8542948ce9886a384b1f97b3a
I20251027 02:48:43.366433 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet-rowsetlayout-svg?id=3b4653bd3b624448801635ef21288354
I20251027 02:48:43.366590 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet-rowsetlayout-svg?id=56a1f0cb4ab34f2fb12c1fadfea103b7
I20251027 02:48:43.366739 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet-rowsetlayout-svg?id=932afee710bd49159618bd88c0295333
I20251027 02:48:43.366891 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet-rowsetlayout-svg?id=ee405edc42594078b91898ef9fbf8b33
I20251027 02:48:43.367044 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet?id=028cac69bd2c4adda9da6b2e07fcee84
I20251027 02:48:43.367225 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet?id=1a5f4ca8542948ce9886a384b1f97b3a
I20251027 02:48:43.367377 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet?id=3b4653bd3b624448801635ef21288354
I20251027 02:48:43.367527 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet?id=56a1f0cb4ab34f2fb12c1fadfea103b7
I20251027 02:48:43.367673 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet?id=932afee710bd49159618bd88c0295333
I20251027 02:48:43.367820 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablet?id=ee405edc42594078b91898ef9fbf8b33
I20251027 02:48:43.367970 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/tablets
I20251027 02:48:43.368122 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz
I20251027 02:48:43.368281 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=acceptor%20pool
I20251027 02:48:43.368435 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=all
I20251027 02:48:43.368585 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=cache
I20251027 02:48:43.368744 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=cloud%20detector
I20251027 02:48:43.368889 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=heartbeater
I20251027 02:48:43.369030 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=kernel-watchdog
I20251027 02:48:43.369192 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=maintenance
I20251027 02:48:43.369351 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=ntp
I20251027 02:48:43.369491 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=reactor
I20251027 02:48:43.369633 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=scanners
I20251027 02:48:43.369771 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=server
I20251027 02:48:43.369918 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:48:43.370061 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:48:43.370204 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:48:43.370376 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=service%20pool%20kudu.tserver.TabletServerAdminService
I20251027 02:48:43.370520 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=service%20pool%20kudu.tserver.TabletServerService
I20251027 02:48:43.370658 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=tablet-copy
I20251027 02:48:43.370801 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/threadz?group=thread%20pool
I20251027 02:48:43.370944 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/transactions
I20251027 02:48:43.371090 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:42311/varz
I20251027 02:48:43.371244 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667
I20251027 02:48:43.371389 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/bootstrap/js/bootstrap-table.min.js
I20251027 02:48:43.371529 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/bootstrap/js/bootstrap.min.js
I20251027 02:48:43.371673 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/config
I20251027 02:48:43.371815 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/dashboards
I20251027 02:48:43.371953 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/healthz
I20251027 02:48:43.372098 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/jquery-3.5.1.min.js
I20251027 02:48:43.372251 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/kudu.js
I20251027 02:48:43.372395 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/log-anchors?id=028cac69bd2c4adda9da6b2e07fcee84
I20251027 02:48:43.372535 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/log-anchors?id=1a5f4ca8542948ce9886a384b1f97b3a
I20251027 02:48:43.372673 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/log-anchors?id=3b4653bd3b624448801635ef21288354
I20251027 02:48:43.372810 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/log-anchors?id=56a1f0cb4ab34f2fb12c1fadfea103b7
I20251027 02:48:43.372968 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/log-anchors?id=932afee710bd49159618bd88c0295333
I20251027 02:48:43.373131 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/log-anchors?id=ee405edc42594078b91898ef9fbf8b33
I20251027 02:48:43.373286 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/logs
I20251027 02:48:43.373420 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/maintenance-manager
I20251027 02:48:43.373555 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/mem-trackers
I20251027 02:48:43.373692 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/memz
I20251027 02:48:43.373826 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/metrics
I20251027 02:48:43.373958 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/metrics_prometheus
I20251027 02:48:43.374094 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/rpcz
I20251027 02:48:43.374233 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/scans
I20251027 02:48:43.374370 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/stacks
I20251027 02:48:43.374502 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/startup
I20251027 02:48:43.374635 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet-consensus-status?id=028cac69bd2c4adda9da6b2e07fcee84
I20251027 02:48:43.374768 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet-consensus-status?id=1a5f4ca8542948ce9886a384b1f97b3a
I20251027 02:48:43.374897 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet-consensus-status?id=3b4653bd3b624448801635ef21288354
I20251027 02:48:43.375029 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet-consensus-status?id=56a1f0cb4ab34f2fb12c1fadfea103b7
I20251027 02:48:43.375164 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet-consensus-status?id=932afee710bd49159618bd88c0295333
I20251027 02:48:43.375308 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet-consensus-status?id=ee405edc42594078b91898ef9fbf8b33
I20251027 02:48:43.375442 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet-rowsetlayout-svg?id=028cac69bd2c4adda9da6b2e07fcee84
I20251027 02:48:43.375578 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet-rowsetlayout-svg?id=1a5f4ca8542948ce9886a384b1f97b3a
I20251027 02:48:43.375712 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet-rowsetlayout-svg?id=3b4653bd3b624448801635ef21288354
I20251027 02:48:43.375849 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet-rowsetlayout-svg?id=56a1f0cb4ab34f2fb12c1fadfea103b7
I20251027 02:48:43.375981 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet-rowsetlayout-svg?id=932afee710bd49159618bd88c0295333
I20251027 02:48:43.376119 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet-rowsetlayout-svg?id=ee405edc42594078b91898ef9fbf8b33
I20251027 02:48:43.376269 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet?id=028cac69bd2c4adda9da6b2e07fcee84
I20251027 02:48:43.376407 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet?id=1a5f4ca8542948ce9886a384b1f97b3a
I20251027 02:48:43.376539 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet?id=3b4653bd3b624448801635ef21288354
I20251027 02:48:43.376672 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet?id=56a1f0cb4ab34f2fb12c1fadfea103b7
I20251027 02:48:43.376801 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet?id=932afee710bd49159618bd88c0295333
I20251027 02:48:43.376928 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablet?id=ee405edc42594078b91898ef9fbf8b33
I20251027 02:48:43.377067 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/tablets
I20251027 02:48:43.377234 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz
I20251027 02:48:43.377377 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=acceptor%20pool
I20251027 02:48:43.377513 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=all
I20251027 02:48:43.377648 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=cache
I20251027 02:48:43.377803 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=cloud%20detector
I20251027 02:48:43.377941 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=heartbeater
I20251027 02:48:43.378079 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=kernel-watchdog
I20251027 02:48:43.378221 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=maintenance
I20251027 02:48:43.378355 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=ntp
I20251027 02:48:43.378484 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=reactor
I20251027 02:48:43.378618 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=scanners
I20251027 02:48:43.378743 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=server
I20251027 02:48:43.378873 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:48:43.379000 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:48:43.379127 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:48:43.379268 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=service%20pool%20kudu.tserver.TabletServerAdminService
I20251027 02:48:43.379395 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=service%20pool%20kudu.tserver.TabletServerService
I20251027 02:48:43.379530 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=tablet-copy
I20251027 02:48:43.379670 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/threadz?group=thread%20pool
I20251027 02:48:43.379808 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/transactions
I20251027 02:48:43.379941 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:38667/varz
I20251027 02:48:43.380079 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715
I20251027 02:48:43.380224 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/bootstrap/js/bootstrap-table.min.js
I20251027 02:48:43.380366 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/bootstrap/js/bootstrap.min.js
I20251027 02:48:43.380504 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/config
I20251027 02:48:43.380641 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/healthz
I20251027 02:48:43.380779 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/ipki-ca-cert
I20251027 02:48:43.380919 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/jquery-3.5.1.min.js
I20251027 02:48:43.381057 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/kudu.js
I20251027 02:48:43.381247 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/logs
I20251027 02:48:43.381390 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/masters
I20251027 02:48:43.381522 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/mem-trackers
I20251027 02:48:43.381659 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/memz
I20251027 02:48:43.381793 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/metrics
I20251027 02:48:43.381927 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/metrics_prometheus
I20251027 02:48:43.382061 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/rpcz
I20251027 02:48:43.382197 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/stacks
I20251027 02:48:43.382365 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/startup
I20251027 02:48:43.382506 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/table?id=a62c2aa9a80d46b990c6068782281e6d
I20251027 02:48:43.382644 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/tables
I20251027 02:48:43.382783 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/tables?&redirects=1
I20251027 02:48:43.382923 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/tablet-servers
I20251027 02:48:43.383061 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz
I20251027 02:48:43.383199 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz?group=acceptor%20pool
I20251027 02:48:43.383349 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz?group=all
I20251027 02:48:43.383491 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz?group=cache
I20251027 02:48:43.383630 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz?group=catalog%20manager
I20251027 02:48:43.383770 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz?group=cloud%20detector
I20251027 02:48:43.383908 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz?group=kernel-watchdog
I20251027 02:48:43.384043 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz?group=maintenance
I20251027 02:48:43.384184 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz?group=master
I20251027 02:48:43.384336 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz?group=ntp
I20251027 02:48:43.384480 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz?group=reactor
I20251027 02:48:43.384621 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz?group=server
I20251027 02:48:43.384760 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:48:43.384907 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz?group=service%20pool%20kudu.master.MasterService
I20251027 02:48:43.385048 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:48:43.385226 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:48:43.385376 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz?group=tablet-copy
I20251027 02:48:43.385522 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/threadz?group=thread%20pool
I20251027 02:48:43.385664 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:38715/varz
I20251027 02:48:43.385798 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759
I20251027 02:48:43.385932 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/bootstrap/js/bootstrap-table.min.js
I20251027 02:48:43.386068 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/bootstrap/js/bootstrap.min.js
I20251027 02:48:43.386199 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/config
I20251027 02:48:43.386324 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/healthz
I20251027 02:48:43.386456 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/ipki-ca-cert
I20251027 02:48:43.386595 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/jquery-3.5.1.min.js
I20251027 02:48:43.386734 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/kudu.js
I20251027 02:48:43.386873 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/logs
I20251027 02:48:43.387008 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/masters
I20251027 02:48:43.387147 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/mem-trackers
I20251027 02:48:43.387295 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/memz
I20251027 02:48:43.387434 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/metrics
I20251027 02:48:43.387564 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/metrics_prometheus
I20251027 02:48:43.387724 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/rpcz
I20251027 02:48:43.387857 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/stacks
I20251027 02:48:43.387989 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/startup
I20251027 02:48:43.388119 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/tables
I20251027 02:48:43.388259 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/tablet-servers
I20251027 02:48:43.388393 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz
I20251027 02:48:43.388530 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz?group=acceptor%20pool
I20251027 02:48:43.388682 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz?group=all
I20251027 02:48:43.388813 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz?group=cache
I20251027 02:48:43.388926 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz?group=catalog%20manager
I20251027 02:48:43.389047 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz?group=cloud%20detector
I20251027 02:48:43.389187 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz?group=kernel-watchdog
I20251027 02:48:43.389331 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz?group=maintenance
I20251027 02:48:43.389467 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz?group=master
I20251027 02:48:43.389602 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz?group=ntp
I20251027 02:48:43.389735 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz?group=reactor
I20251027 02:48:43.389871 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz?group=server
I20251027 02:48:43.390023 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:48:43.390159 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz?group=service%20pool%20kudu.master.MasterService
I20251027 02:48:43.390303 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:48:43.390442 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:48:43.390579 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz?group=tablet-copy
I20251027 02:48:43.390713 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/threadz?group=thread%20pool
I20251027 02:48:43.390849 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:46759/varz
I20251027 02:48:43.390985 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631
I20251027 02:48:43.391122 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/bootstrap/js/bootstrap-table.min.js
I20251027 02:48:43.391268 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/bootstrap/js/bootstrap.min.js
I20251027 02:48:43.391403 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/config
I20251027 02:48:43.391541 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/healthz
I20251027 02:48:43.391674 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/ipki-ca-cert
I20251027 02:48:43.391821 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/jquery-3.5.1.min.js
I20251027 02:48:43.391957 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/kudu.js
I20251027 02:48:43.392098 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/logs
I20251027 02:48:43.392242 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/masters
I20251027 02:48:43.392377 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/mem-trackers
I20251027 02:48:43.392508 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/memz
I20251027 02:48:43.392640 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/metrics
I20251027 02:48:43.392773 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/metrics_prometheus
I20251027 02:48:43.392905 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/rpcz
I20251027 02:48:43.393038 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/stacks
I20251027 02:48:43.393195 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/startup
I20251027 02:48:43.393359 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/tables
I20251027 02:48:43.393498 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/tablet-servers
I20251027 02:48:43.393635 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz
I20251027 02:48:43.393771 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz?group=acceptor%20pool
I20251027 02:48:43.393908 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz?group=all
I20251027 02:48:43.394048 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz?group=cache
I20251027 02:48:43.394184 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz?group=catalog%20manager
I20251027 02:48:43.394333 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz?group=cloud%20detector
I20251027 02:48:43.394472 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz?group=kernel-watchdog
I20251027 02:48:43.394613 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz?group=maintenance
I20251027 02:48:43.394747 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz?group=master
I20251027 02:48:43.394883 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz?group=ntp
I20251027 02:48:43.395018 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz?group=reactor
I20251027 02:48:43.395159 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz?group=server
I20251027 02:48:43.395313 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:48:43.395443 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz?group=service%20pool%20kudu.master.MasterService
I20251027 02:48:43.395573 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:48:43.395721 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:48:43.395846 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz?group=tablet-copy
I20251027 02:48:43.395970 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/threadz?group=thread%20pool
I20251027 02:48:43.396095 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:37631/varz
I20251027 02:48:43.400584 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1730012
I20251027 02:48:43.460224 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1730206
I20251027 02:48:43.524014 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1730357
W20251027 02:48:43.523967 1730426 connection.cc:537] client connection to 127.152.231.194:35939 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251027 02:48:43.586231 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1729485
I20251027 02:48:43.632114 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1729565
I20251027 02:48:43.681286 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1729887
2025-10-27T02:48:43Z chronyd exiting
[ OK ] WebserverCrawlITest.TestAllWebPages/0 (80196 ms)
[ RUN ] WebserverCrawlITest.TestAllWebPages/1
I20251027 02:48:43.786299 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
2025-10-27T02:48:43Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-10-27T02:48:43Z Disabled control of system clock
I20251027 02:48:43.837102 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:48:43.837796 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:48:43.838264 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:48:43.842178 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.254:40859
--webserver_interface=127.152.231.254
--webserver_port=0
--builtin_ntp_servers=127.152.231.212:40625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.152.231.254:40859,127.152.231.253:45409,127.152.231.252:45143
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www with env {}
W20251027 02:48:44.194525 1732069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:48:44.195097 1732069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:48:44.195538 1732069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:48:44.228415 1732069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251027 02:48:44.228706 1732069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:48:44.228905 1732069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251027 02:48:44.229117 1732069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251027 02:48:44.265872 1732069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:40625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.152.231.254:40859,127.152.231.253:45409,127.152.231.252:45143
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.254:40859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-0/data/info.pb
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:48:44.267274 1732069 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:48:44.268937 1732069 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:48:44.283815 1732080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:48:44.285430 1732081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:48:44.291298 1732083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:48:45.703920 1732082 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20251027 02:48:45.704047 1732069 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:48:45.708771 1732069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:48:45.713146 1732069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:48:45.714653 1732069 hybrid_clock.cc:648] HybridClock initialized: now 1761533325714569 us; error 74 us; skew 500 ppm
I20251027 02:48:45.715767 1732069 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:48:45.723835 1732069 webserver.cc:492] Webserver started at http://127.152.231.254:45271/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:48:45.725139 1732069 fs_manager.cc:362] Metadata directory not provided
I20251027 02:48:45.725430 1732069 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:48:45.726038 1732069 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:48:45.733698 1732069 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-0/data/instance:
uuid: "d494a45cd725401aaf93b2113952bb18"
format_stamp: "Formatted at 2025-10-27 02:48:45 on 5fd53c4cbb9d"
I20251027 02:48:45.735190 1732069 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-0/wal/instance:
uuid: "d494a45cd725401aaf93b2113952bb18"
format_stamp: "Formatted at 2025-10-27 02:48:45 on 5fd53c4cbb9d"
I20251027 02:48:45.745877 1732069 fs_manager.cc:696] Time spent creating directory manager: real 0.010s user 0.010s sys 0.001s
I20251027 02:48:45.753638 1732226 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:48:45.755121 1732069 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20251027 02:48:45.755518 1732069 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-0/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-0/wal
uuid: "d494a45cd725401aaf93b2113952bb18"
format_stamp: "Formatted at 2025-10-27 02:48:45 on 5fd53c4cbb9d"
I20251027 02:48:45.755932 1732069 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-0/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-0/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:48:45.855028 1732069 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:48:45.856705 1732069 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:48:45.857287 1732069 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:48:45.973719 1732069 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.254:40859
I20251027 02:48:45.973881 1732277 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.254:40859 every 8 connection(s)
I20251027 02:48:45.977221 1732069 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-0/data/info.pb
I20251027 02:48:45.981815 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1732069
I20251027 02:48:45.982347 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-0/wal/instance
I20251027 02:48:45.985967 1732278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:45.986013 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.253:45409
--webserver_interface=127.152.231.253
--webserver_port=0
--builtin_ntp_servers=127.152.231.212:40625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.152.231.254:40859,127.152.231.253:45409,127.152.231.252:45143
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www with env {}
I20251027 02:48:46.004514 1732278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:48:46.037784 1732278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } has no permanent_uuid. Determining permanent_uuid...
W20251027 02:48:46.040988 1732230 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.152.231.253:45409: connect: Connection refused (error 111)
W20251027 02:48:46.047116 1732278 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:45409: Network error: Client connection negotiation failed: client connection to 127.152.231.253:45409: connect: Connection refused (error 111)
I20251027 02:48:46.096700 1732278 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } attempt: 1
W20251027 02:48:46.101202 1732278 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:45409: Network error: Client connection negotiation failed: client connection to 127.152.231.253:45409: connect: Connection refused (error 111)
I20251027 02:48:46.169660 1732278 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } attempt: 2
W20251027 02:48:46.174400 1732278 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:45409: Network error: Client connection negotiation failed: client connection to 127.152.231.253:45409: connect: Connection refused (error 111)
I20251027 02:48:46.265843 1732278 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } attempt: 3
W20251027 02:48:46.270304 1732278 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:45409: Network error: Client connection negotiation failed: client connection to 127.152.231.253:45409: connect: Connection refused (error 111)
I20251027 02:48:46.413808 1732278 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } attempt: 4
W20251027 02:48:46.418458 1732278 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:45409: Network error: Client connection negotiation failed: client connection to 127.152.231.253:45409: connect: Connection refused (error 111)
W20251027 02:48:46.510795 1732280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:48:46.511619 1732280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:48:46.512524 1732280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:48:46.572476 1732280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251027 02:48:46.572922 1732280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:48:46.573334 1732280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251027 02:48:46.573698 1732280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251027 02:48:46.642311 1732280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:40625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.152.231.254:40859,127.152.231.253:45409,127.152.231.252:45143
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.253:45409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-1/data/info.pb
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.253
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:48:46.644315 1732280 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:48:46.646909 1732280 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:48:46.665380 1732304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:48:46.717917 1732278 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } attempt: 5
W20251027 02:48:46.723354 1732278 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:45409: Network error: Client connection negotiation failed: client connection to 127.152.231.253:45409: connect: Connection refused (error 111)
I20251027 02:48:47.270843 1732278 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } attempt: 6
W20251027 02:48:47.279908 1732278 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:45409: Network error: Client connection negotiation failed: client connection to 127.152.231.253:45409: connect: Connection refused (error 111)
W20251027 02:48:48.066010 1732303 debug-util.cc:398] Leaking SignalData structure 0x7b0800037520 after lost signal to thread 1732280
W20251027 02:48:46.666240 1732305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:48:48.249284 1732280 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.584s user 0.721s sys 0.862s
W20251027 02:48:48.249823 1732280 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.584s user 0.721s sys 0.863s
W20251027 02:48:48.251948 1732307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:48:48.254685 1732306 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1584 milliseconds
I20251027 02:48:48.254813 1732280 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:48:48.256508 1732280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:48:48.260669 1732280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:48:48.262147 1732280 hybrid_clock.cc:648] HybridClock initialized: now 1761533328262083 us; error 46 us; skew 500 ppm
I20251027 02:48:48.263351 1732280 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:48:48.271394 1732280 webserver.cc:492] Webserver started at http://127.152.231.253:41229/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:48:48.272826 1732280 fs_manager.cc:362] Metadata directory not provided
I20251027 02:48:48.273151 1732280 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:48:48.273808 1732280 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:48:48.281934 1732280 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-1/data/instance:
uuid: "e2eece34c28441a2bc5d6c84db4e45cd"
format_stamp: "Formatted at 2025-10-27 02:48:48 on 5fd53c4cbb9d"
I20251027 02:48:48.283648 1732280 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-1/wal/instance:
uuid: "e2eece34c28441a2bc5d6c84db4e45cd"
format_stamp: "Formatted at 2025-10-27 02:48:48 on 5fd53c4cbb9d"
I20251027 02:48:48.294785 1732280 fs_manager.cc:696] Time spent creating directory manager: real 0.010s user 0.012s sys 0.000s
I20251027 02:48:48.303458 1732316 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:48:48.304847 1732280 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20251027 02:48:48.305327 1732280 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-1/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-1/wal
uuid: "e2eece34c28441a2bc5d6c84db4e45cd"
format_stamp: "Formatted at 2025-10-27 02:48:48 on 5fd53c4cbb9d"
I20251027 02:48:48.305797 1732280 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-1/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-1/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:48:48.340441 1732278 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } attempt: 7
W20251027 02:48:48.348141 1732278 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:45409: Network error: Client connection negotiation failed: client connection to 127.152.231.253:45409: connect: Connection refused (error 111)
I20251027 02:48:48.428468 1732280 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:48:48.430511 1732280 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:48:48.431200 1732280 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:48:48.555359 1732280 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.253:45409
I20251027 02:48:48.555400 1732369 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.253:45409 every 8 connection(s)
I20251027 02:48:48.559381 1732280 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-1/data/info.pb
I20251027 02:48:48.560668 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1732280
I20251027 02:48:48.561206 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-1/wal/instance
I20251027 02:48:48.564917 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.252:45143
--webserver_interface=127.152.231.252
--webserver_port=0
--builtin_ntp_servers=127.152.231.212:40625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.152.231.254:40859,127.152.231.253:45409,127.152.231.252:45143
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www with env {}
I20251027 02:48:48.567011 1732370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:48.579242 1732370 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:48:48.604104 1732370 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:48:48.623808 1732370 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } has no permanent_uuid. Determining permanent_uuid...
W20251027 02:48:48.626791 1732320 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.152.231.252:45143: connect: Connection refused (error 111)
W20251027 02:48:48.634199 1732370 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:45143: Network error: Client connection negotiation failed: client connection to 127.152.231.252:45143: connect: Connection refused (error 111)
I20251027 02:48:48.683777 1732370 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } attempt: 1
W20251027 02:48:48.688334 1732370 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:45143: Network error: Client connection negotiation failed: client connection to 127.152.231.252:45143: connect: Connection refused (error 111)
I20251027 02:48:48.756781 1732370 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } attempt: 2
W20251027 02:48:48.761379 1732370 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:45143: Network error: Client connection negotiation failed: client connection to 127.152.231.252:45143: connect: Connection refused (error 111)
I20251027 02:48:48.852840 1732370 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } attempt: 3
W20251027 02:48:48.857311 1732370 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:45143: Network error: Client connection negotiation failed: client connection to 127.152.231.252:45143: connect: Connection refused (error 111)
I20251027 02:48:49.000757 1732370 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } attempt: 4
W20251027 02:48:49.006465 1732370 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:45143: Network error: Client connection negotiation failed: client connection to 127.152.231.252:45143: connect: Connection refused (error 111)
W20251027 02:48:49.106362 1732372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:48:49.107213 1732372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:48:49.108078 1732372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:48:49.169528 1732372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251027 02:48:49.169982 1732372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:48:49.170337 1732372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251027 02:48:49.170691 1732372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251027 02:48:49.239336 1732372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:40625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.152.231.254:40859,127.152.231.253:45409,127.152.231.252:45143
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.252:45143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-2/data/info.pb
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.252
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:48:49.241365 1732372 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:48:49.243893 1732372 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:48:49.261842 1732385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:48:49.305928 1732370 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } attempt: 5
W20251027 02:48:49.310542 1732370 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:45143: Network error: Client connection negotiation failed: client connection to 127.152.231.252:45143: connect: Connection refused (error 111)
I20251027 02:48:49.857993 1732370 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } attempt: 6
W20251027 02:48:49.866572 1732370 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:45143: Network error: Client connection negotiation failed: client connection to 127.152.231.252:45143: connect: Connection refused (error 111)
I20251027 02:48:50.438642 1732278 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } attempt: 8
I20251027 02:48:50.450038 1732278 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } has no permanent_uuid. Determining permanent_uuid...
W20251027 02:48:50.456462 1732278 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:45143: Network error: Client connection negotiation failed: client connection to 127.152.231.252:45143: connect: Connection refused (error 111)
I20251027 02:48:50.521963 1732278 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } attempt: 1
W20251027 02:48:50.527098 1732278 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:45143: Network error: Client connection negotiation failed: client connection to 127.152.231.252:45143: connect: Connection refused (error 111)
I20251027 02:48:50.580579 1732278 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } attempt: 2
W20251027 02:48:50.585414 1732278 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:45143: Network error: Client connection negotiation failed: client connection to 127.152.231.252:45143: connect: Connection refused (error 111)
I20251027 02:48:50.661880 1732278 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } attempt: 3
W20251027 02:48:50.666899 1732278 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:45143: Network error: Client connection negotiation failed: client connection to 127.152.231.252:45143: connect: Connection refused (error 111)
I20251027 02:48:50.822381 1732278 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } attempt: 4
W20251027 02:48:50.826875 1732278 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:45143: Network error: Client connection negotiation failed: client connection to 127.152.231.252:45143: connect: Connection refused (error 111)
W20251027 02:48:49.262748 1732386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:48:50.846478 1732388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:48:50.849205 1732387 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1580 milliseconds
I20251027 02:48:50.849318 1732372 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:48:50.850878 1732372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:48:50.855178 1732372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:48:50.856671 1732372 hybrid_clock.cc:648] HybridClock initialized: now 1761533330856613 us; error 50 us; skew 500 ppm
I20251027 02:48:50.857858 1732372 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:48:50.865988 1732372 webserver.cc:492] Webserver started at http://127.152.231.252:44417/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:48:50.867264 1732372 fs_manager.cc:362] Metadata directory not provided
I20251027 02:48:50.867534 1732372 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:48:50.868142 1732372 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:48:50.875663 1732372 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-2/data/instance:
uuid: "0457e7bb62854dc2be53f752b6d401e4"
format_stamp: "Formatted at 2025-10-27 02:48:50 on 5fd53c4cbb9d"
I20251027 02:48:50.877235 1732372 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-2/wal/instance:
uuid: "0457e7bb62854dc2be53f752b6d401e4"
format_stamp: "Formatted at 2025-10-27 02:48:50 on 5fd53c4cbb9d"
I20251027 02:48:50.888012 1732372 fs_manager.cc:696] Time spent creating directory manager: real 0.010s user 0.001s sys 0.008s
I20251027 02:48:50.895697 1732414 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:48:50.897194 1732372 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20251027 02:48:50.897692 1732372 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-2/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-2/wal
uuid: "0457e7bb62854dc2be53f752b6d401e4"
format_stamp: "Formatted at 2025-10-27 02:48:50 on 5fd53c4cbb9d"
I20251027 02:48:50.898159 1732372 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-2/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-2/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:48:50.927078 1732370 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } attempt: 7
W20251027 02:48:50.936023 1732370 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:45143: Network error: Client connection negotiation failed: client connection to 127.152.231.252:45143: connect: Connection refused (error 111)
I20251027 02:48:50.979835 1732372 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:48:50.982147 1732372 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:48:50.982776 1732372 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:48:51.099351 1732372 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.252:45143
I20251027 02:48:51.099527 1732467 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.252:45143 every 8 connection(s)
I20251027 02:48:51.102926 1732372 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-2/data/info.pb
I20251027 02:48:51.110836 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1732372
I20251027 02:48:51.111419 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/master-2/wal/instance
I20251027 02:48:51.111603 1732468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:51.123333 1732278 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } attempt: 5
I20251027 02:48:51.130643 1732468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:48:51.158176 1732468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:48:51.161387 1732278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18: Bootstrap starting.
I20251027 02:48:51.167888 1732278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:51.169543 1732468 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:48:51.169879 1732278 log.cc:826] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18: Log is configured to *not* fsync() on all Append() calls
I20251027 02:48:51.175920 1732278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18: No bootstrap required, opened a new log
I20251027 02:48:51.189563 1732468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4: Bootstrap starting.
I20251027 02:48:51.197417 1732468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:51.199494 1732468 log.cc:826] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4: Log is configured to *not* fsync() on all Append() calls
I20251027 02:48:51.202718 1732278 raft_consensus.cc:359] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } }
I20251027 02:48:51.204135 1732278 raft_consensus.cc:385] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:51.204419 1732278 raft_consensus.cc:740] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d494a45cd725401aaf93b2113952bb18, State: Initialized, Role: FOLLOWER
I20251027 02:48:51.205794 1732468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4: No bootstrap required, opened a new log
I20251027 02:48:51.205327 1732278 consensus_queue.cc:260] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } }
I20251027 02:48:51.208576 1732478 sys_catalog.cc:455] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } } }
I20251027 02:48:51.209529 1732478 sys_catalog.cc:458] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:48:51.210338 1732278 sys_catalog.cc:565] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [sys.catalog]: configured and running, proceeding with master startup.
I20251027 02:48:51.232290 1732468 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } }
I20251027 02:48:51.233824 1732468 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:51.234128 1732468 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0457e7bb62854dc2be53f752b6d401e4, State: Initialized, Role: FOLLOWER
I20251027 02:48:51.235109 1732468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } }
I20251027 02:48:51.238859 1732490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } } }
I20251027 02:48:51.240427 1732490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4 [sys.catalog]: This master's current role is: FOLLOWER
W20251027 02:48:51.241251 1732422 tablet.cc:2378] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:48:51.241290 1732468 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4 [sys.catalog]: configured and running, proceeding with master startup.
W20251027 02:48:51.245841 1732489 catalog_manager.cc:1568] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251027 02:48:51.246220 1732489 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251027 02:48:51.242524 1732069 master_runner.cc:186] Error getting master registration for 127.152.231.253:45409: OK, instance_id { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" instance_seqno: 1761533328521325 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20251027 02:48:51.269448 1732069 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20251027 02:48:51.276727 1732502 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251027 02:48:51.277153 1732502 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251027 02:48:51.277801 1732372 master_runner.cc:186] Error getting master registration for 127.152.231.253:45409: OK, instance_id { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" instance_seqno: 1761533328521325 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20251027 02:48:51.307664 1732372 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20251027 02:48:51.373068 1732232 tablet.cc:2378] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:48:51.502465 1732478 raft_consensus.cc:493] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:48:51.503129 1732478 raft_consensus.cc:515] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } }
I20251027 02:48:51.506065 1732478 leader_election.cc:290] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2eece34c28441a2bc5d6c84db4e45cd (127.152.231.253:45409), 0457e7bb62854dc2be53f752b6d401e4 (127.152.231.252:45143)
I20251027 02:48:51.507560 1732443 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d494a45cd725401aaf93b2113952bb18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0457e7bb62854dc2be53f752b6d401e4" is_pre_election: true
I20251027 02:48:51.509438 1732443 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d494a45cd725401aaf93b2113952bb18 in term 0.
I20251027 02:48:51.511044 1732228 leader_election.cc:304] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0457e7bb62854dc2be53f752b6d401e4, d494a45cd725401aaf93b2113952bb18; no voters:
I20251027 02:48:51.512018 1732478 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:48:51.512441 1732478 raft_consensus.cc:493] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:48:51.512818 1732478 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:51.520802 1732478 raft_consensus.cc:515] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } }
I20251027 02:48:51.522835 1732478 leader_election.cc:290] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [CANDIDATE]: Term 1 election: Requested vote from peers e2eece34c28441a2bc5d6c84db4e45cd (127.152.231.253:45409), 0457e7bb62854dc2be53f752b6d401e4 (127.152.231.252:45143)
I20251027 02:48:51.523988 1732443 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d494a45cd725401aaf93b2113952bb18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0457e7bb62854dc2be53f752b6d401e4"
I20251027 02:48:51.524544 1732443 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:51.531935 1732443 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d494a45cd725401aaf93b2113952bb18 in term 1.
I20251027 02:48:51.533166 1732228 leader_election.cc:304] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0457e7bb62854dc2be53f752b6d401e4, d494a45cd725401aaf93b2113952bb18; no voters:
I20251027 02:48:51.533980 1732478 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:48:51.535854 1732478 raft_consensus.cc:697] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [term 1 LEADER]: Becoming Leader. State: Replica: d494a45cd725401aaf93b2113952bb18, State: Running, Role: LEADER
I20251027 02:48:51.536823 1732478 consensus_queue.cc:237] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } }
I20251027 02:48:51.550113 1732504 sys_catalog.cc:455] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d494a45cd725401aaf93b2113952bb18. Latest consensus state: current_term: 1 leader_uuid: "d494a45cd725401aaf93b2113952bb18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } } }
I20251027 02:48:51.551033 1732504 sys_catalog.cc:458] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [sys.catalog]: This master's current role is: LEADER
I20251027 02:48:51.554033 1732506 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251027 02:48:51.563890 1732506 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251027 02:48:51.582880 1732443 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4 [term 1 FOLLOWER]: Refusing update from remote peer d494a45cd725401aaf93b2113952bb18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:48:51.585280 1732478 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:48:51.605132 1732490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d494a45cd725401aaf93b2113952bb18. Latest consensus state: current_term: 1 leader_uuid: "d494a45cd725401aaf93b2113952bb18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } } }
I20251027 02:48:51.606539 1732490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:48:51.607388 1732504 sys_catalog.cc:455] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d494a45cd725401aaf93b2113952bb18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } } }
I20251027 02:48:51.608297 1732504 sys_catalog.cc:458] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [sys.catalog]: This master's current role is: LEADER
I20251027 02:48:51.610352 1732506 catalog_manager.cc:1357] Generated new cluster ID: 3acd0e71388b4746bd5e17b0d15df649
I20251027 02:48:51.610627 1732506 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251027 02:48:51.611366 1732490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d494a45cd725401aaf93b2113952bb18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } } }
I20251027 02:48:51.612030 1732490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:48:51.635015 1732506 catalog_manager.cc:1380] Generated new certificate authority record
I20251027 02:48:51.637250 1732506 catalog_manager.cc:1514] Loading token signing keys...
I20251027 02:48:51.661240 1732506 catalog_manager.cc:6022] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18: Generated new TSK 0
I20251027 02:48:51.662473 1732506 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251027 02:48:51.677452 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:48:51.679774 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.152.231.193:0
--local_ip_for_outbound_sockets=127.152.231.193
--webserver_interface=127.152.231.193
--webserver_port=0
--tserver_master_addrs=127.152.231.254:40859,127.152.231.253:45409,127.152.231.252:45143
--builtin_ntp_servers=127.152.231.212:40625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www with env {}
W20251027 02:48:52.223320 1732513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:48:52.224050 1732513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:48:52.224961 1732513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20251027 02:48:52.283136 1732502 catalog_manager.cc:1269] Loaded cluster ID: 3acd0e71388b4746bd5e17b0d15df649
I20251027 02:48:52.283447 1732502 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4: loading cluster ID for follower catalog manager: success
W20251027 02:48:52.287942 1732513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:48:52.289625 1732513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.152.231.193
I20251027 02:48:52.290874 1732502 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4: acquiring CA information for follower catalog manager: success
I20251027 02:48:52.295915 1732502 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 0457e7bb62854dc2be53f752b6d401e4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251027 02:48:52.358234 1732513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:40625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.152.231.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-0/data/info.pb
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.193
--webserver_port=0
--tserver_master_addrs=127.152.231.254:40859,127.152.231.253:45409,127.152.231.252:45143
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.152.231.193
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:48:52.360232 1732513 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:48:52.362839 1732513 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:48:52.383924 1732520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:48:53.026495 1732370 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } attempt: 8
I20251027 02:48:53.052853 1732370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd: Bootstrap starting.
I20251027 02:48:53.059883 1732370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:53.061945 1732370 log.cc:826] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd: Log is configured to *not* fsync() on all Append() calls
I20251027 02:48:53.067655 1732370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd: No bootstrap required, opened a new log
I20251027 02:48:53.094981 1732370 raft_consensus.cc:359] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } }
I20251027 02:48:53.095898 1732370 raft_consensus.cc:385] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:53.096196 1732370 raft_consensus.cc:740] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2eece34c28441a2bc5d6c84db4e45cd, State: Initialized, Role: FOLLOWER
I20251027 02:48:53.097189 1732370 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } }
I20251027 02:48:53.100967 1732529 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } } }
I20251027 02:48:53.102460 1732529 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:48:53.102877 1732370 sys_catalog.cc:565] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd [sys.catalog]: configured and running, proceeding with master startup.
I20251027 02:48:53.103063 1732345 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d494a45cd725401aaf93b2113952bb18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" is_pre_election: true
I20251027 02:48:53.103085 1732344 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d494a45cd725401aaf93b2113952bb18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2eece34c28441a2bc5d6c84db4e45cd"
I20251027 02:48:53.104075 1732343 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd [term 0 FOLLOWER]: Advancing to term 1
W20251027 02:48:53.104966 1732344 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50224 (request call id 3) took 1581 ms (client timeout 1942 ms). Trace:
W20251027 02:48:53.105055 1732345 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50224 (request call id 2) took 1598 ms (client timeout 1714 ms). Trace:
W20251027 02:48:53.105418 1732344 rpcz_store.cc:269] 1027 02:48:51.523312 (+ 0us) service_pool.cc:168] Inserting onto call queue
1027 02:48:51.523449 (+ 137us) service_pool.cc:225] Handling call
1027 02:48:53.104886 (+1581437us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20251027 02:48:53.105499 1732345 rpcz_store.cc:269] 1027 02:48:51.506637 (+ 0us) service_pool.cc:168] Inserting onto call queue
1027 02:48:51.506847 (+ 210us) service_pool.cc:225] Handling call
1027 02:48:53.105000 (+1598153us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20251027 02:48:53.112164 1732343 raft_consensus.cc:1275] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd [term 1 FOLLOWER]: Refusing update from remote peer d494a45cd725401aaf93b2113952bb18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:48:53.112883 1732343 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50224 (request call id 4) took 1530 ms. Trace:
I20251027 02:48:53.113983 1732343 rpcz_store.cc:276] 1027 02:48:51.582105 (+ 0us) service_pool.cc:168] Inserting onto call queue
1027 02:48:51.582260 (+ 155us) service_pool.cc:225] Handling call
1027 02:48:53.103252 (+1520992us) raft_consensus.cc:1471] Updating replica for 0 ops
1027 02:48:53.112504 (+ 9252us) raft_consensus.cc:1697] Filling consensus response to leader.
1027 02:48:53.112825 (+ 321us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20251027 02:48:53.115397 1732531 consensus_queue.cc:1048] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:48:53.147857 1732529 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader d494a45cd725401aaf93b2113952bb18. Latest consensus state: current_term: 1 leader_uuid: "d494a45cd725401aaf93b2113952bb18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } } }
I20251027 02:48:53.148383 1732529 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:48:53.149457 1732543 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d494a45cd725401aaf93b2113952bb18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } } }
I20251027 02:48:53.150987 1732543 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:48:53.161597 1732547 catalog_manager.cc:1269] Loaded cluster ID: 3acd0e71388b4746bd5e17b0d15df649
I20251027 02:48:53.161880 1732547 catalog_manager.cc:1562] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd: loading cluster ID for follower catalog manager: success
I20251027 02:48:53.168962 1732547 catalog_manager.cc:1584] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd: acquiring CA information for follower catalog manager: success
I20251027 02:48:53.174685 1732547 catalog_manager.cc:1612] T 00000000000000000000000000000000 P e2eece34c28441a2bc5d6c84db4e45cd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251027 02:48:53.702736 1732531 sys_catalog.cc:455] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d494a45cd725401aaf93b2113952bb18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d494a45cd725401aaf93b2113952bb18" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 40859 } } peers { permanent_uuid: "e2eece34c28441a2bc5d6c84db4e45cd" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 45409 } } peers { permanent_uuid: "0457e7bb62854dc2be53f752b6d401e4" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 45143 } } }
I20251027 02:48:53.704246 1732531 sys_catalog.cc:458] T 00000000000000000000000000000000 P d494a45cd725401aaf93b2113952bb18 [sys.catalog]: This master's current role is: LEADER
W20251027 02:48:53.784046 1732519 debug-util.cc:398] Leaking SignalData structure 0x7b0800037940 after lost signal to thread 1732513
W20251027 02:48:52.385411 1732521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:48:53.901185 1732513 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.517s user 0.686s sys 0.831s
W20251027 02:48:53.901733 1732513 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.518s user 0.686s sys 0.831s
W20251027 02:48:53.904772 1732523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:48:53.907238 1732522 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1517 milliseconds
I20251027 02:48:53.907359 1732513 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:48:53.908990 1732513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:48:53.912830 1732513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:48:53.914350 1732513 hybrid_clock.cc:648] HybridClock initialized: now 1761533333914300 us; error 50 us; skew 500 ppm
I20251027 02:48:53.915541 1732513 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:48:53.923931 1732513 webserver.cc:492] Webserver started at http://127.152.231.193:32903/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:48:53.925310 1732513 fs_manager.cc:362] Metadata directory not provided
I20251027 02:48:53.925619 1732513 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:48:53.926277 1732513 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:48:53.934201 1732513 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-0/data/instance:
uuid: "aa4328f7b8d74ea3aef5aa96cbf87961"
format_stamp: "Formatted at 2025-10-27 02:48:53 on 5fd53c4cbb9d"
I20251027 02:48:53.935933 1732513 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-0/wal/instance:
uuid: "aa4328f7b8d74ea3aef5aa96cbf87961"
format_stamp: "Formatted at 2025-10-27 02:48:53 on 5fd53c4cbb9d"
I20251027 02:48:53.947449 1732513 fs_manager.cc:696] Time spent creating directory manager: real 0.011s user 0.010s sys 0.003s
I20251027 02:48:53.955950 1732558 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:48:53.957458 1732513 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.005s sys 0.002s
I20251027 02:48:53.957937 1732513 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-0/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-0/wal
uuid: "aa4328f7b8d74ea3aef5aa96cbf87961"
format_stamp: "Formatted at 2025-10-27 02:48:53 on 5fd53c4cbb9d"
I20251027 02:48:53.958410 1732513 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:48:54.054831 1732513 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:48:54.056759 1732513 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:48:54.057448 1732513 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:48:54.061735 1732513 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251027 02:48:54.069698 1732513 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251027 02:48:54.070021 1732513 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251027 02:48:54.070358 1732513 ts_tablet_manager.cc:616] Registered 0 tablets
I20251027 02:48:54.070576 1732513 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251027 02:48:54.305263 1732513 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.193:34633
I20251027 02:48:54.305441 1732671 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.193:34633 every 8 connection(s)
I20251027 02:48:54.308964 1732513 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-0/data/info.pb
I20251027 02:48:54.312234 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1732513
I20251027 02:48:54.312984 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-0/wal/instance
I20251027 02:48:54.324647 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:48:54.326958 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.152.231.194:0
--local_ip_for_outbound_sockets=127.152.231.194
--webserver_interface=127.152.231.194
--webserver_port=0
--tserver_master_addrs=127.152.231.254:40859,127.152.231.253:45409,127.152.231.252:45143
--builtin_ntp_servers=127.152.231.212:40625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www with env {}
I20251027 02:48:54.346904 1732672 heartbeater.cc:344] Connected to a master server at 127.152.231.252:45143
I20251027 02:48:54.347469 1732672 heartbeater.cc:461] Registering TS with master...
I20251027 02:48:54.347987 1732673 heartbeater.cc:344] Connected to a master server at 127.152.231.253:45409
I20251027 02:48:54.348377 1732673 heartbeater.cc:461] Registering TS with master...
I20251027 02:48:54.348999 1732672 heartbeater.cc:507] Master 127.152.231.252:45143 requested a full tablet report, sending...
I20251027 02:48:54.349823 1732673 heartbeater.cc:507] Master 127.152.231.253:45409 requested a full tablet report, sending...
I20251027 02:48:54.352662 1732674 heartbeater.cc:344] Connected to a master server at 127.152.231.254:40859
I20251027 02:48:54.352816 1732433 ts_manager.cc:194] Registered new tserver with Master: aa4328f7b8d74ea3aef5aa96cbf87961 (127.152.231.193:34633)
I20251027 02:48:54.352945 1732335 ts_manager.cc:194] Registered new tserver with Master: aa4328f7b8d74ea3aef5aa96cbf87961 (127.152.231.193:34633)
I20251027 02:48:54.353068 1732674 heartbeater.cc:461] Registering TS with master...
I20251027 02:48:54.354122 1732674 heartbeater.cc:507] Master 127.152.231.254:40859 requested a full tablet report, sending...
I20251027 02:48:54.357273 1732242 ts_manager.cc:194] Registered new tserver with Master: aa4328f7b8d74ea3aef5aa96cbf87961 (127.152.231.193:34633)
I20251027 02:48:54.360759 1732242 master_service.cc:502] Signed X509 certificate for tserver {username='jenkins-slave'} at 127.152.231.193:35421
W20251027 02:48:54.733462 1732682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:48:54.733948 1732682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:48:54.734442 1732682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:48:54.766806 1732682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:48:54.767647 1732682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.152.231.194
I20251027 02:48:54.803701 1732682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:40625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.152.231.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-1/data/info.pb
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.194
--webserver_port=0
--tserver_master_addrs=127.152.231.254:40859,127.152.231.253:45409,127.152.231.252:45143
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.152.231.194
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:48:54.805083 1732682 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:48:54.806826 1732682 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:48:54.822646 1732689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:48:55.365427 1732674 heartbeater.cc:499] Master 127.152.231.254:40859 was elected leader, sending a full tablet report...
W20251027 02:48:54.824031 1732690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:48:55.976039 1732692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:48:55.977221 1732691 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1149 milliseconds
I20251027 02:48:55.977342 1732682 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:48:55.978915 1732682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:48:55.982600 1732682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:48:55.984094 1732682 hybrid_clock.cc:648] HybridClock initialized: now 1761533335984022 us; error 43 us; skew 500 ppm
I20251027 02:48:55.985353 1732682 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:48:55.993214 1732682 webserver.cc:492] Webserver started at http://127.152.231.194:46827/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:48:55.994531 1732682 fs_manager.cc:362] Metadata directory not provided
I20251027 02:48:55.994818 1732682 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:48:55.995409 1732682 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:48:56.003270 1732682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-1/data/instance:
uuid: "407cabaabd22488a986388278be5c13f"
format_stamp: "Formatted at 2025-10-27 02:48:55 on 5fd53c4cbb9d"
I20251027 02:48:56.004777 1732682 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-1/wal/instance:
uuid: "407cabaabd22488a986388278be5c13f"
format_stamp: "Formatted at 2025-10-27 02:48:55 on 5fd53c4cbb9d"
I20251027 02:48:56.015250 1732682 fs_manager.cc:696] Time spent creating directory manager: real 0.010s user 0.005s sys 0.004s
I20251027 02:48:56.023087 1732700 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:48:56.024399 1732682 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20251027 02:48:56.024817 1732682 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-1/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-1/wal
uuid: "407cabaabd22488a986388278be5c13f"
format_stamp: "Formatted at 2025-10-27 02:48:55 on 5fd53c4cbb9d"
I20251027 02:48:56.025321 1732682 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:48:56.104060 1732682 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:48:56.105376 1732682 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:48:56.105839 1732682 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:48:56.108678 1732682 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251027 02:48:56.113981 1732682 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251027 02:48:56.114171 1732682 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251027 02:48:56.114388 1732682 ts_tablet_manager.cc:616] Registered 0 tablets
I20251027 02:48:56.114506 1732682 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251027 02:48:56.282837 1732682 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.194:45037
I20251027 02:48:56.282950 1732821 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.194:45037 every 8 connection(s)
I20251027 02:48:56.285215 1732682 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-1/data/info.pb
I20251027 02:48:56.295215 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1732682
I20251027 02:48:56.295687 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-1/wal/instance
I20251027 02:48:56.302178 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:48:56.304237 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.152.231.195:0
--local_ip_for_outbound_sockets=127.152.231.195
--webserver_interface=127.152.231.195
--webserver_port=0
--tserver_master_addrs=127.152.231.254:40859,127.152.231.253:45409,127.152.231.252:45143
--builtin_ntp_servers=127.152.231.212:40625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www with env {}
I20251027 02:48:56.316277 1732823 heartbeater.cc:344] Connected to a master server at 127.152.231.253:45409
I20251027 02:48:56.316349 1732822 heartbeater.cc:344] Connected to a master server at 127.152.231.252:45143
I20251027 02:48:56.316674 1732823 heartbeater.cc:461] Registering TS with master...
I20251027 02:48:56.317112 1732822 heartbeater.cc:461] Registering TS with master...
I20251027 02:48:56.317607 1732824 heartbeater.cc:344] Connected to a master server at 127.152.231.254:40859
I20251027 02:48:56.317936 1732823 heartbeater.cc:507] Master 127.152.231.253:45409 requested a full tablet report, sending...
I20251027 02:48:56.318105 1732824 heartbeater.cc:461] Registering TS with master...
I20251027 02:48:56.318390 1732822 heartbeater.cc:507] Master 127.152.231.252:45143 requested a full tablet report, sending...
I20251027 02:48:56.319139 1732824 heartbeater.cc:507] Master 127.152.231.254:40859 requested a full tablet report, sending...
I20251027 02:48:56.320699 1732335 ts_manager.cc:194] Registered new tserver with Master: 407cabaabd22488a986388278be5c13f (127.152.231.194:45037)
I20251027 02:48:56.321174 1732242 ts_manager.cc:194] Registered new tserver with Master: 407cabaabd22488a986388278be5c13f (127.152.231.194:45037)
I20251027 02:48:56.321357 1732432 ts_manager.cc:194] Registered new tserver with Master: 407cabaabd22488a986388278be5c13f (127.152.231.194:45037)
I20251027 02:48:56.322816 1732242 master_service.cc:502] Signed X509 certificate for tserver {username='jenkins-slave'} at 127.152.231.194:57461
W20251027 02:48:56.638104 1732832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:48:56.638595 1732832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:48:56.639065 1732832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:48:56.671105 1732832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:48:56.671937 1732832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.152.231.195
I20251027 02:48:56.707422 1732832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:40625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.152.231.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-2/data/info.pb
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.195
--webserver_port=0
--tserver_master_addrs=127.152.231.254:40859,127.152.231.253:45409,127.152.231.252:45143
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.152.231.195
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:48:56.708788 1732832 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:48:56.710474 1732832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:48:56.724339 1732843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:48:57.325771 1732824 heartbeater.cc:499] Master 127.152.231.254:40859 was elected leader, sending a full tablet report...
W20251027 02:48:56.725893 1732844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:48:57.886190 1732846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:48:57.887511 1732845 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1158 milliseconds
W20251027 02:48:57.888455 1732832 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.164s user 0.507s sys 0.656s
W20251027 02:48:57.888695 1732832 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.164s user 0.507s sys 0.656s
I20251027 02:48:57.888875 1732832 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:48:57.890013 1732832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:48:57.892804 1732832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:48:57.894129 1732832 hybrid_clock.cc:648] HybridClock initialized: now 1761533337894083 us; error 38 us; skew 500 ppm
I20251027 02:48:57.894948 1732832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:48:57.901458 1732832 webserver.cc:492] Webserver started at http://127.152.231.195:46309/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:48:57.902349 1732832 fs_manager.cc:362] Metadata directory not provided
I20251027 02:48:57.902540 1732832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:48:57.902933 1732832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:48:57.908476 1732832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-2/data/instance:
uuid: "04a5ea4feab74c3480f93fc3b9182d69"
format_stamp: "Formatted at 2025-10-27 02:48:57 on 5fd53c4cbb9d"
I20251027 02:48:57.909590 1732832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-2/wal/instance:
uuid: "04a5ea4feab74c3480f93fc3b9182d69"
format_stamp: "Formatted at 2025-10-27 02:48:57 on 5fd53c4cbb9d"
I20251027 02:48:57.917548 1732832 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20251027 02:48:57.923779 1732854 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:48:57.924860 1732832 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.005s sys 0.001s
I20251027 02:48:57.925164 1732832 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-2/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-2/wal
uuid: "04a5ea4feab74c3480f93fc3b9182d69"
format_stamp: "Formatted at 2025-10-27 02:48:57 on 5fd53c4cbb9d"
I20251027 02:48:57.925474 1732832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:48:58.008491 1732832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:48:58.009760 1732832 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:48:58.010255 1732832 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:48:58.013402 1732832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251027 02:48:58.018460 1732832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251027 02:48:58.018651 1732832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251027 02:48:58.018837 1732832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251027 02:48:58.018959 1732832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251027 02:48:58.183162 1732832 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.195:46433
I20251027 02:48:58.183300 1732966 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.195:46433 every 8 connection(s)
I20251027 02:48:58.185413 1732832 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-2/data/info.pb
I20251027 02:48:58.191982 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1732832
I20251027 02:48:58.192487 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_1.1761533243417696-1729439-0/minicluster-data/ts-2/wal/instance
I20251027 02:48:58.216223 1732967 heartbeater.cc:344] Connected to a master server at 127.152.231.252:45143
I20251027 02:48:58.216667 1732967 heartbeater.cc:461] Registering TS with master...
I20251027 02:48:58.217806 1732969 heartbeater.cc:344] Connected to a master server at 127.152.231.254:40859
I20251027 02:48:58.217891 1732967 heartbeater.cc:507] Master 127.152.231.252:45143 requested a full tablet report, sending...
I20251027 02:48:58.218149 1732969 heartbeater.cc:461] Registering TS with master...
I20251027 02:48:58.219151 1732969 heartbeater.cc:507] Master 127.152.231.254:40859 requested a full tablet report, sending...
I20251027 02:48:58.219508 1732968 heartbeater.cc:344] Connected to a master server at 127.152.231.253:45409
I20251027 02:48:58.219781 1732968 heartbeater.cc:461] Registering TS with master...
I20251027 02:48:58.220436 1732968 heartbeater.cc:507] Master 127.152.231.253:45409 requested a full tablet report, sending...
I20251027 02:48:58.220976 1732432 ts_manager.cc:194] Registered new tserver with Master: 04a5ea4feab74c3480f93fc3b9182d69 (127.152.231.195:46433)
I20251027 02:48:58.221246 1732242 ts_manager.cc:194] Registered new tserver with Master: 04a5ea4feab74c3480f93fc3b9182d69 (127.152.231.195:46433)
I20251027 02:48:58.222929 1732242 master_service.cc:502] Signed X509 certificate for tserver {username='jenkins-slave'} at 127.152.231.195:45877
I20251027 02:48:58.223294 1732335 ts_manager.cc:194] Registered new tserver with Master: 04a5ea4feab74c3480f93fc3b9182d69 (127.152.231.195:46433)
I20251027 02:48:58.232761 1729439 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251027 02:48:58.243116 1729439 test_util.cc:276] Using random seed: -253571053
I20251027 02:48:58.278034 1732242 catalog_manager.cc:2257] Servicing CreateTable request from {username='jenkins-slave'} at 127.0.0.1:35072:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\004\001\000UUU\025\004\001\000\252\252\252*\004\001\000\377\377\377?\004\001\000TUUU\004\001\000\251\252\252j"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251027 02:48:58.282717 1732242 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251027 02:48:58.391306 1732748 tablet_service.cc:1505] Processing CreateTablet for tablet a6ec10f95955403fac0d633b40dcc3f9 (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION VALUES < 357913941
I20251027 02:48:58.391191 1732745 tablet_service.cc:1505] Processing CreateTablet for tablet 643c36a0f0bc44309a0962ddc5c54437 (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20251027 02:48:58.391296 1732747 tablet_service.cc:1505] Processing CreateTablet for tablet e377623c494144c38bc40e664d53b097 (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20251027 02:48:58.390450 1732744 tablet_service.cc:1505] Processing CreateTablet for tablet 43d4caa6e0d54e0391ade495eb2ac2f0 (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20251027 02:48:58.390362 1732743 tablet_service.cc:1505] Processing CreateTablet for tablet 1036261eea4842f8b5274de21215f71e (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20251027 02:48:58.390535 1732746 tablet_service.cc:1505] Processing CreateTablet for tablet dde9c836fd1a46ba83a3358453abbfb8 (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20251027 02:48:58.393482 1732748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6ec10f95955403fac0d633b40dcc3f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.394236 1732747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e377623c494144c38bc40e664d53b097. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.394869 1732744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43d4caa6e0d54e0391ade495eb2ac2f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.395725 1732746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dde9c836fd1a46ba83a3358453abbfb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.396437 1732743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1036261eea4842f8b5274de21215f71e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.396852 1732745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 643c36a0f0bc44309a0962ddc5c54437. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.401552 1732602 tablet_service.cc:1505] Processing CreateTablet for tablet 43d4caa6e0d54e0391ade495eb2ac2f0 (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20251027 02:48:58.400658 1732601 tablet_service.cc:1505] Processing CreateTablet for tablet 1036261eea4842f8b5274de21215f71e (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20251027 02:48:58.401400 1732604 tablet_service.cc:1505] Processing CreateTablet for tablet dde9c836fd1a46ba83a3358453abbfb8 (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20251027 02:48:58.401461 1732603 tablet_service.cc:1505] Processing CreateTablet for tablet 643c36a0f0bc44309a0962ddc5c54437 (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20251027 02:48:58.401561 1732607 tablet_service.cc:1505] Processing CreateTablet for tablet a6ec10f95955403fac0d633b40dcc3f9 (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION VALUES < 357913941
I20251027 02:48:58.400672 1732606 tablet_service.cc:1505] Processing CreateTablet for tablet e377623c494144c38bc40e664d53b097 (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20251027 02:48:58.402554 1732902 tablet_service.cc:1505] Processing CreateTablet for tablet a6ec10f95955403fac0d633b40dcc3f9 (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION VALUES < 357913941
I20251027 02:48:58.401612 1732899 tablet_service.cc:1505] Processing CreateTablet for tablet 643c36a0f0bc44309a0962ddc5c54437 (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20251027 02:48:58.402606 1732900 tablet_service.cc:1505] Processing CreateTablet for tablet dde9c836fd1a46ba83a3358453abbfb8 (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20251027 02:48:58.403957 1732607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6ec10f95955403fac0d633b40dcc3f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.401772 1732898 tablet_service.cc:1505] Processing CreateTablet for tablet 43d4caa6e0d54e0391ade495eb2ac2f0 (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20251027 02:48:58.402500 1732901 tablet_service.cc:1505] Processing CreateTablet for tablet e377623c494144c38bc40e664d53b097 (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20251027 02:48:58.401666 1732897 tablet_service.cc:1505] Processing CreateTablet for tablet 1036261eea4842f8b5274de21215f71e (DEFAULT_TABLE table=test-workload [id=844ef6d9458447798a2dfb2a6088af3f]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20251027 02:48:58.404608 1732602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43d4caa6e0d54e0391ade495eb2ac2f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.405128 1732601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1036261eea4842f8b5274de21215f71e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.405750 1732606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e377623c494144c38bc40e664d53b097. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.406051 1732899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 643c36a0f0bc44309a0962ddc5c54437. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.406488 1732603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 643c36a0f0bc44309a0962ddc5c54437. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.406665 1732902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6ec10f95955403fac0d633b40dcc3f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.407248 1732604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dde9c836fd1a46ba83a3358453abbfb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.407243 1732900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dde9c836fd1a46ba83a3358453abbfb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.407860 1732901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e377623c494144c38bc40e664d53b097. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.408613 1732897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1036261eea4842f8b5274de21215f71e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.409287 1732898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43d4caa6e0d54e0391ade495eb2ac2f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:48:58.423902 1733001 tablet_bootstrap.cc:492] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f: Bootstrap starting.
I20251027 02:48:58.430281 1733002 tablet_bootstrap.cc:492] T dde9c836fd1a46ba83a3358453abbfb8 P aa4328f7b8d74ea3aef5aa96cbf87961: Bootstrap starting.
I20251027 02:48:58.430830 1733001 tablet_bootstrap.cc:654] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.432433 1733001 log.cc:826] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f: Log is configured to *not* fsync() on all Append() calls
I20251027 02:48:58.436599 1733004 tablet_bootstrap.cc:492] T a6ec10f95955403fac0d633b40dcc3f9 P 04a5ea4feab74c3480f93fc3b9182d69: Bootstrap starting.
I20251027 02:48:58.437067 1733002 tablet_bootstrap.cc:654] T dde9c836fd1a46ba83a3358453abbfb8 P aa4328f7b8d74ea3aef5aa96cbf87961: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.437580 1733001 tablet_bootstrap.cc:492] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f: No bootstrap required, opened a new log
I20251027 02:48:58.438036 1733001 ts_tablet_manager.cc:1403] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f: Time spent bootstrapping tablet: real 0.015s user 0.013s sys 0.000s
I20251027 02:48:58.438617 1733002 log.cc:826] T dde9c836fd1a46ba83a3358453abbfb8 P aa4328f7b8d74ea3aef5aa96cbf87961: Log is configured to *not* fsync() on all Append() calls
I20251027 02:48:58.442777 1733004 tablet_bootstrap.cc:654] T a6ec10f95955403fac0d633b40dcc3f9 P 04a5ea4feab74c3480f93fc3b9182d69: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.444288 1733004 log.cc:826] T a6ec10f95955403fac0d633b40dcc3f9 P 04a5ea4feab74c3480f93fc3b9182d69: Log is configured to *not* fsync() on all Append() calls
I20251027 02:48:58.444538 1733002 tablet_bootstrap.cc:492] T dde9c836fd1a46ba83a3358453abbfb8 P aa4328f7b8d74ea3aef5aa96cbf87961: No bootstrap required, opened a new log
I20251027 02:48:58.444988 1733002 ts_tablet_manager.cc:1403] T dde9c836fd1a46ba83a3358453abbfb8 P aa4328f7b8d74ea3aef5aa96cbf87961: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.007s
I20251027 02:48:58.449977 1733004 tablet_bootstrap.cc:492] T a6ec10f95955403fac0d633b40dcc3f9 P 04a5ea4feab74c3480f93fc3b9182d69: No bootstrap required, opened a new log
I20251027 02:48:58.450388 1733004 ts_tablet_manager.cc:1403] T a6ec10f95955403fac0d633b40dcc3f9 P 04a5ea4feab74c3480f93fc3b9182d69: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.001s
I20251027 02:48:58.455667 1733001 raft_consensus.cc:359] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.456353 1733001 raft_consensus.cc:385] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.456600 1733001 raft_consensus.cc:740] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 407cabaabd22488a986388278be5c13f, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.457636 1733001 consensus_queue.cc:260] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.460430 1733001 ts_tablet_manager.cc:1434] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f: Time spent starting tablet: real 0.022s user 0.018s sys 0.004s
I20251027 02:48:58.461509 1733001 tablet_bootstrap.cc:492] T 643c36a0f0bc44309a0962ddc5c54437 P 407cabaabd22488a986388278be5c13f: Bootstrap starting.
I20251027 02:48:58.462692 1733002 raft_consensus.cc:359] T dde9c836fd1a46ba83a3358453abbfb8 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.463342 1733002 raft_consensus.cc:385] T dde9c836fd1a46ba83a3358453abbfb8 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.463551 1733002 raft_consensus.cc:740] T dde9c836fd1a46ba83a3358453abbfb8 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa4328f7b8d74ea3aef5aa96cbf87961, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.464193 1733002 consensus_queue.cc:260] T dde9c836fd1a46ba83a3358453abbfb8 P aa4328f7b8d74ea3aef5aa96cbf87961 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.464885 1733001 tablet_bootstrap.cc:654] T 643c36a0f0bc44309a0962ddc5c54437 P 407cabaabd22488a986388278be5c13f: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.467396 1733002 ts_tablet_manager.cc:1434] T dde9c836fd1a46ba83a3358453abbfb8 P aa4328f7b8d74ea3aef5aa96cbf87961: Time spent starting tablet: real 0.022s user 0.020s sys 0.001s
I20251027 02:48:58.468086 1733004 raft_consensus.cc:359] T a6ec10f95955403fac0d633b40dcc3f9 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.468616 1733002 tablet_bootstrap.cc:492] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961: Bootstrap starting.
I20251027 02:48:58.468678 1733004 raft_consensus.cc:385] T a6ec10f95955403fac0d633b40dcc3f9 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.468947 1733004 raft_consensus.cc:740] T a6ec10f95955403fac0d633b40dcc3f9 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a5ea4feab74c3480f93fc3b9182d69, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.469656 1733004 consensus_queue.cc:260] T a6ec10f95955403fac0d633b40dcc3f9 P 04a5ea4feab74c3480f93fc3b9182d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.471573 1733001 tablet_bootstrap.cc:492] T 643c36a0f0bc44309a0962ddc5c54437 P 407cabaabd22488a986388278be5c13f: No bootstrap required, opened a new log
I20251027 02:48:58.471896 1733001 ts_tablet_manager.cc:1403] T 643c36a0f0bc44309a0962ddc5c54437 P 407cabaabd22488a986388278be5c13f: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20251027 02:48:58.472004 1733002 tablet_bootstrap.cc:654] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.472363 1732969 heartbeater.cc:499] Master 127.152.231.254:40859 was elected leader, sending a full tablet report...
I20251027 02:48:58.472898 1733004 ts_tablet_manager.cc:1434] T a6ec10f95955403fac0d633b40dcc3f9 P 04a5ea4feab74c3480f93fc3b9182d69: Time spent starting tablet: real 0.022s user 0.011s sys 0.011s
I20251027 02:48:58.473629 1733001 raft_consensus.cc:359] T 643c36a0f0bc44309a0962ddc5c54437 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.473861 1733004 tablet_bootstrap.cc:492] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 04a5ea4feab74c3480f93fc3b9182d69: Bootstrap starting.
I20251027 02:48:58.474047 1733001 raft_consensus.cc:385] T 643c36a0f0bc44309a0962ddc5c54437 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.474291 1733001 raft_consensus.cc:740] T 643c36a0f0bc44309a0962ddc5c54437 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 407cabaabd22488a986388278be5c13f, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.474795 1733001 consensus_queue.cc:260] T 643c36a0f0bc44309a0962ddc5c54437 P 407cabaabd22488a986388278be5c13f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.476809 1733001 ts_tablet_manager.cc:1434] T 643c36a0f0bc44309a0962ddc5c54437 P 407cabaabd22488a986388278be5c13f: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20251027 02:48:58.477633 1733001 tablet_bootstrap.cc:492] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f: Bootstrap starting.
I20251027 02:48:58.478885 1733004 tablet_bootstrap.cc:654] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 04a5ea4feab74c3480f93fc3b9182d69: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.479214 1733002 tablet_bootstrap.cc:492] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961: No bootstrap required, opened a new log
I20251027 02:48:58.479600 1733002 ts_tablet_manager.cc:1403] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20251027 02:48:58.481453 1733001 tablet_bootstrap.cc:654] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.481873 1733002 raft_consensus.cc:359] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.482622 1733002 raft_consensus.cc:385] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.482887 1733002 raft_consensus.cc:740] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa4328f7b8d74ea3aef5aa96cbf87961, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.483371 1733002 consensus_queue.cc:260] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.485098 1733002 ts_tablet_manager.cc:1434] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20251027 02:48:58.485652 1733004 tablet_bootstrap.cc:492] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 04a5ea4feab74c3480f93fc3b9182d69: No bootstrap required, opened a new log
I20251027 02:48:58.485769 1733002 tablet_bootstrap.cc:492] T 643c36a0f0bc44309a0962ddc5c54437 P aa4328f7b8d74ea3aef5aa96cbf87961: Bootstrap starting.
I20251027 02:48:58.485934 1733004 ts_tablet_manager.cc:1403] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 04a5ea4feab74c3480f93fc3b9182d69: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20251027 02:48:58.487264 1733001 tablet_bootstrap.cc:492] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f: No bootstrap required, opened a new log
I20251027 02:48:58.487541 1733001 ts_tablet_manager.cc:1403] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20251027 02:48:58.487485 1733004 raft_consensus.cc:359] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.487983 1733004 raft_consensus.cc:385] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.488211 1733004 raft_consensus.cc:740] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a5ea4feab74c3480f93fc3b9182d69, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.488660 1733004 consensus_queue.cc:260] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 04a5ea4feab74c3480f93fc3b9182d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.489059 1733002 tablet_bootstrap.cc:654] T 643c36a0f0bc44309a0962ddc5c54437 P aa4328f7b8d74ea3aef5aa96cbf87961: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.489159 1733001 raft_consensus.cc:359] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.489593 1733001 raft_consensus.cc:385] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.489745 1733001 raft_consensus.cc:740] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 407cabaabd22488a986388278be5c13f, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.490257 1733004 ts_tablet_manager.cc:1434] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 04a5ea4feab74c3480f93fc3b9182d69: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20251027 02:48:58.490150 1733001 consensus_queue.cc:260] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.491062 1733004 tablet_bootstrap.cc:492] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69: Bootstrap starting.
I20251027 02:48:58.491670 1733001 ts_tablet_manager.cc:1434] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20251027 02:48:58.492549 1733001 tablet_bootstrap.cc:492] T a6ec10f95955403fac0d633b40dcc3f9 P 407cabaabd22488a986388278be5c13f: Bootstrap starting.
I20251027 02:48:58.495010 1733004 tablet_bootstrap.cc:654] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.495088 1733002 tablet_bootstrap.cc:492] T 643c36a0f0bc44309a0962ddc5c54437 P aa4328f7b8d74ea3aef5aa96cbf87961: No bootstrap required, opened a new log
I20251027 02:48:58.495523 1733002 ts_tablet_manager.cc:1403] T 643c36a0f0bc44309a0962ddc5c54437 P aa4328f7b8d74ea3aef5aa96cbf87961: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.001s
I20251027 02:48:58.496304 1733001 tablet_bootstrap.cc:654] T a6ec10f95955403fac0d633b40dcc3f9 P 407cabaabd22488a986388278be5c13f: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.497820 1733002 raft_consensus.cc:359] T 643c36a0f0bc44309a0962ddc5c54437 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.498466 1733002 raft_consensus.cc:385] T 643c36a0f0bc44309a0962ddc5c54437 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.498732 1733002 raft_consensus.cc:740] T 643c36a0f0bc44309a0962ddc5c54437 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa4328f7b8d74ea3aef5aa96cbf87961, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.499320 1733002 consensus_queue.cc:260] T 643c36a0f0bc44309a0962ddc5c54437 P aa4328f7b8d74ea3aef5aa96cbf87961 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.499982 1733007 raft_consensus.cc:493] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:48:58.501313 1733004 tablet_bootstrap.cc:492] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69: No bootstrap required, opened a new log
I20251027 02:48:58.501303 1733002 ts_tablet_manager.cc:1434] T 643c36a0f0bc44309a0962ddc5c54437 P aa4328f7b8d74ea3aef5aa96cbf87961: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20251027 02:48:58.501593 1733004 ts_tablet_manager.cc:1403] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20251027 02:48:58.502136 1733002 tablet_bootstrap.cc:492] T 43d4caa6e0d54e0391ade495eb2ac2f0 P aa4328f7b8d74ea3aef5aa96cbf87961: Bootstrap starting.
I20251027 02:48:58.501267 1733007 raft_consensus.cc:515] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.503273 1733001 tablet_bootstrap.cc:492] T a6ec10f95955403fac0d633b40dcc3f9 P 407cabaabd22488a986388278be5c13f: No bootstrap required, opened a new log
I20251027 02:48:58.503119 1733004 raft_consensus.cc:359] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.503527 1733004 raft_consensus.cc:385] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.503226 1733017 raft_consensus.cc:493] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:48:58.503638 1733001 ts_tablet_manager.cc:1403] T a6ec10f95955403fac0d633b40dcc3f9 P 407cabaabd22488a986388278be5c13f: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20251027 02:48:58.503681 1733004 raft_consensus.cc:740] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a5ea4feab74c3480f93fc3b9182d69, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.504062 1733004 consensus_queue.cc:260] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.503821 1733017 raft_consensus.cc:515] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.505331 1733004 ts_tablet_manager.cc:1434] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20251027 02:48:58.505985 1733004 tablet_bootstrap.cc:492] T dde9c836fd1a46ba83a3358453abbfb8 P 04a5ea4feab74c3480f93fc3b9182d69: Bootstrap starting.
I20251027 02:48:58.505892 1733001 raft_consensus.cc:359] T a6ec10f95955403fac0d633b40dcc3f9 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.506381 1733001 raft_consensus.cc:385] T a6ec10f95955403fac0d633b40dcc3f9 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.506394 1733007 leader_election.cc:290] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa4328f7b8d74ea3aef5aa96cbf87961 (127.152.231.193:34633), 04a5ea4feab74c3480f93fc3b9182d69 (127.152.231.195:46433)
I20251027 02:48:58.506677 1733001 raft_consensus.cc:740] T a6ec10f95955403fac0d633b40dcc3f9 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 407cabaabd22488a986388278be5c13f, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.507169 1733002 tablet_bootstrap.cc:654] T 43d4caa6e0d54e0391ade495eb2ac2f0 P aa4328f7b8d74ea3aef5aa96cbf87961: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.507175 1733001 consensus_queue.cc:260] T a6ec10f95955403fac0d633b40dcc3f9 P 407cabaabd22488a986388278be5c13f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.507274 1733017 leader_election.cc:290] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa4328f7b8d74ea3aef5aa96cbf87961 (127.152.231.193:34633), 04a5ea4feab74c3480f93fc3b9182d69 (127.152.231.195:46433)
I20251027 02:48:58.508927 1733001 ts_tablet_manager.cc:1434] T a6ec10f95955403fac0d633b40dcc3f9 P 407cabaabd22488a986388278be5c13f: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251027 02:48:58.510141 1733001 tablet_bootstrap.cc:492] T 1036261eea4842f8b5274de21215f71e P 407cabaabd22488a986388278be5c13f: Bootstrap starting.
I20251027 02:48:58.510810 1733004 tablet_bootstrap.cc:654] T dde9c836fd1a46ba83a3358453abbfb8 P 04a5ea4feab74c3480f93fc3b9182d69: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.514315 1733002 tablet_bootstrap.cc:492] T 43d4caa6e0d54e0391ade495eb2ac2f0 P aa4328f7b8d74ea3aef5aa96cbf87961: No bootstrap required, opened a new log
I20251027 02:48:58.514745 1733002 ts_tablet_manager.cc:1403] T 43d4caa6e0d54e0391ade495eb2ac2f0 P aa4328f7b8d74ea3aef5aa96cbf87961: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20251027 02:48:58.516965 1733001 tablet_bootstrap.cc:654] T 1036261eea4842f8b5274de21215f71e P 407cabaabd22488a986388278be5c13f: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.517016 1733002 raft_consensus.cc:359] T 43d4caa6e0d54e0391ade495eb2ac2f0 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.517587 1733004 tablet_bootstrap.cc:492] T dde9c836fd1a46ba83a3358453abbfb8 P 04a5ea4feab74c3480f93fc3b9182d69: No bootstrap required, opened a new log
I20251027 02:48:58.517653 1733002 raft_consensus.cc:385] T 43d4caa6e0d54e0391ade495eb2ac2f0 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.517951 1733002 raft_consensus.cc:740] T 43d4caa6e0d54e0391ade495eb2ac2f0 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa4328f7b8d74ea3aef5aa96cbf87961, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.518103 1733004 ts_tablet_manager.cc:1403] T dde9c836fd1a46ba83a3358453abbfb8 P 04a5ea4feab74c3480f93fc3b9182d69: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20251027 02:48:58.518540 1733002 consensus_queue.cc:260] T 43d4caa6e0d54e0391ade495eb2ac2f0 P aa4328f7b8d74ea3aef5aa96cbf87961 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.520527 1733002 ts_tablet_manager.cc:1434] T 43d4caa6e0d54e0391ade495eb2ac2f0 P aa4328f7b8d74ea3aef5aa96cbf87961: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20251027 02:48:58.520325 1733004 raft_consensus.cc:359] T dde9c836fd1a46ba83a3358453abbfb8 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.520852 1733004 raft_consensus.cc:385] T dde9c836fd1a46ba83a3358453abbfb8 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.521023 1733004 raft_consensus.cc:740] T dde9c836fd1a46ba83a3358453abbfb8 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a5ea4feab74c3480f93fc3b9182d69, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.521474 1733002 tablet_bootstrap.cc:492] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961: Bootstrap starting.
I20251027 02:48:58.521545 1733004 consensus_queue.cc:260] T dde9c836fd1a46ba83a3358453abbfb8 P 04a5ea4feab74c3480f93fc3b9182d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.523506 1733004 ts_tablet_manager.cc:1434] T dde9c836fd1a46ba83a3358453abbfb8 P 04a5ea4feab74c3480f93fc3b9182d69: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20251027 02:48:58.524338 1733004 tablet_bootstrap.cc:492] T 1036261eea4842f8b5274de21215f71e P 04a5ea4feab74c3480f93fc3b9182d69: Bootstrap starting.
I20251027 02:48:58.525054 1732922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "43d4caa6e0d54e0391ade495eb2ac2f0" candidate_uuid: "407cabaabd22488a986388278be5c13f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a5ea4feab74c3480f93fc3b9182d69" is_pre_election: true
I20251027 02:48:58.525686 1733001 tablet_bootstrap.cc:492] T 1036261eea4842f8b5274de21215f71e P 407cabaabd22488a986388278be5c13f: No bootstrap required, opened a new log
I20251027 02:48:58.525475 1732921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dde9c836fd1a46ba83a3358453abbfb8" candidate_uuid: "407cabaabd22488a986388278be5c13f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a5ea4feab74c3480f93fc3b9182d69" is_pre_election: true
I20251027 02:48:58.526113 1733001 ts_tablet_manager.cc:1403] T 1036261eea4842f8b5274de21215f71e P 407cabaabd22488a986388278be5c13f: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.003s
I20251027 02:48:58.526274 1732922 raft_consensus.cc:2468] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 407cabaabd22488a986388278be5c13f in term 0.
I20251027 02:48:58.526388 1732921 raft_consensus.cc:2468] T dde9c836fd1a46ba83a3358453abbfb8 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 407cabaabd22488a986388278be5c13f in term 0.
I20251027 02:48:58.524932 1732627 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "43d4caa6e0d54e0391ade495eb2ac2f0" candidate_uuid: "407cabaabd22488a986388278be5c13f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" is_pre_election: true
I20251027 02:48:58.525540 1732626 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dde9c836fd1a46ba83a3358453abbfb8" candidate_uuid: "407cabaabd22488a986388278be5c13f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" is_pre_election: true
I20251027 02:48:58.527761 1732627 raft_consensus.cc:2468] T 43d4caa6e0d54e0391ade495eb2ac2f0 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 407cabaabd22488a986388278be5c13f in term 0.
I20251027 02:48:58.527791 1732626 raft_consensus.cc:2468] T dde9c836fd1a46ba83a3358453abbfb8 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 407cabaabd22488a986388278be5c13f in term 0.
I20251027 02:48:58.527866 1732703 leader_election.cc:304] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a5ea4feab74c3480f93fc3b9182d69, 407cabaabd22488a986388278be5c13f; no voters:
I20251027 02:48:58.528172 1733002 tablet_bootstrap.cc:654] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.528822 1733017 raft_consensus.cc:2804] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:48:58.529050 1733010 raft_consensus.cc:493] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:48:58.529016 1732703 leader_election.cc:304] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a5ea4feab74c3480f93fc3b9182d69, 407cabaabd22488a986388278be5c13f; no voters:
I20251027 02:48:58.529214 1733017 raft_consensus.cc:493] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:48:58.529035 1733001 raft_consensus.cc:359] T 1036261eea4842f8b5274de21215f71e P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.529697 1733017 raft_consensus.cc:3060] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.529821 1733004 tablet_bootstrap.cc:654] T 1036261eea4842f8b5274de21215f71e P 04a5ea4feab74c3480f93fc3b9182d69: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.529798 1733001 raft_consensus.cc:385] T 1036261eea4842f8b5274de21215f71e P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.529851 1733007 raft_consensus.cc:2804] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:48:58.530117 1733001 raft_consensus.cc:740] T 1036261eea4842f8b5274de21215f71e P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 407cabaabd22488a986388278be5c13f, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.530239 1733007 raft_consensus.cc:493] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:48:58.530524 1733007 raft_consensus.cc:3060] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.529557 1733010 raft_consensus.cc:515] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.530777 1733001 consensus_queue.cc:260] T 1036261eea4842f8b5274de21215f71e P 407cabaabd22488a986388278be5c13f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.534264 1733002 tablet_bootstrap.cc:492] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961: No bootstrap required, opened a new log
I20251027 02:48:58.534709 1733002 ts_tablet_manager.cc:1403] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20251027 02:48:58.534973 1733010 leader_election.cc:290] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa4328f7b8d74ea3aef5aa96cbf87961 (127.152.231.193:34633), 407cabaabd22488a986388278be5c13f (127.152.231.194:45037)
I20251027 02:48:58.536041 1733001 ts_tablet_manager.cc:1434] T 1036261eea4842f8b5274de21215f71e P 407cabaabd22488a986388278be5c13f: Time spent starting tablet: real 0.010s user 0.005s sys 0.003s
I20251027 02:48:58.536867 1733001 tablet_bootstrap.cc:492] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f: Bootstrap starting.
I20251027 02:48:58.537267 1733002 raft_consensus.cc:359] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.537918 1733002 raft_consensus.cc:385] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.538108 1733004 tablet_bootstrap.cc:492] T 1036261eea4842f8b5274de21215f71e P 04a5ea4feab74c3480f93fc3b9182d69: No bootstrap required, opened a new log
I20251027 02:48:58.537881 1733017 raft_consensus.cc:515] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.538194 1733002 raft_consensus.cc:740] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa4328f7b8d74ea3aef5aa96cbf87961, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.538443 1733004 ts_tablet_manager.cc:1403] T 1036261eea4842f8b5274de21215f71e P 04a5ea4feab74c3480f93fc3b9182d69: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.002s
I20251027 02:48:58.538460 1733007 raft_consensus.cc:515] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.538823 1733002 consensus_queue.cc:260] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.539701 1733007 leader_election.cc:290] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [CANDIDATE]: Term 1 election: Requested vote from peers aa4328f7b8d74ea3aef5aa96cbf87961 (127.152.231.193:34633), 04a5ea4feab74c3480f93fc3b9182d69 (127.152.231.195:46433)
I20251027 02:48:58.539719 1733017 leader_election.cc:290] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [CANDIDATE]: Term 1 election: Requested vote from peers aa4328f7b8d74ea3aef5aa96cbf87961 (127.152.231.193:34633), 04a5ea4feab74c3480f93fc3b9182d69 (127.152.231.195:46433)
I20251027 02:48:58.540434 1732626 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "43d4caa6e0d54e0391ade495eb2ac2f0" candidate_uuid: "407cabaabd22488a986388278be5c13f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961"
I20251027 02:48:58.540619 1732921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dde9c836fd1a46ba83a3358453abbfb8" candidate_uuid: "407cabaabd22488a986388278be5c13f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a5ea4feab74c3480f93fc3b9182d69"
I20251027 02:48:58.540709 1732627 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dde9c836fd1a46ba83a3358453abbfb8" candidate_uuid: "407cabaabd22488a986388278be5c13f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961"
I20251027 02:48:58.540889 1733002 ts_tablet_manager.cc:1434] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20251027 02:48:58.540984 1732626 raft_consensus.cc:3060] T 43d4caa6e0d54e0391ade495eb2ac2f0 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.541164 1732922 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "43d4caa6e0d54e0391ade495eb2ac2f0" candidate_uuid: "407cabaabd22488a986388278be5c13f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a5ea4feab74c3480f93fc3b9182d69"
I20251027 02:48:58.541286 1732627 raft_consensus.cc:3060] T dde9c836fd1a46ba83a3358453abbfb8 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.541283 1732921 raft_consensus.cc:3060] T dde9c836fd1a46ba83a3358453abbfb8 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.541321 1733004 raft_consensus.cc:359] T 1036261eea4842f8b5274de21215f71e P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.541685 1732922 raft_consensus.cc:3060] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.541960 1733002 tablet_bootstrap.cc:492] T e377623c494144c38bc40e664d53b097 P aa4328f7b8d74ea3aef5aa96cbf87961: Bootstrap starting.
I20251027 02:48:58.541986 1733004 raft_consensus.cc:385] T 1036261eea4842f8b5274de21215f71e P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.542207 1733004 raft_consensus.cc:740] T 1036261eea4842f8b5274de21215f71e P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a5ea4feab74c3480f93fc3b9182d69, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.542763 1733001 tablet_bootstrap.cc:654] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.543133 1733004 consensus_queue.cc:260] T 1036261eea4842f8b5274de21215f71e P 04a5ea4feab74c3480f93fc3b9182d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.545617 1733004 ts_tablet_manager.cc:1434] T 1036261eea4842f8b5274de21215f71e P 04a5ea4feab74c3480f93fc3b9182d69: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20251027 02:48:58.545415 1732625 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e377623c494144c38bc40e664d53b097" candidate_uuid: "04a5ea4feab74c3480f93fc3b9182d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" is_pre_election: true
I20251027 02:48:58.546392 1733004 tablet_bootstrap.cc:492] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69: Bootstrap starting.
I20251027 02:48:58.548204 1732774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e377623c494144c38bc40e664d53b097" candidate_uuid: "04a5ea4feab74c3480f93fc3b9182d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "407cabaabd22488a986388278be5c13f" is_pre_election: true
W20251027 02:48:58.549311 1732858 leader_election.cc:343] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aa4328f7b8d74ea3aef5aa96cbf87961 (127.152.231.193:34633): Illegal state: must be running to vote when last-logged opid is not known
I20251027 02:48:58.549975 1733001 tablet_bootstrap.cc:492] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f: No bootstrap required, opened a new log
I20251027 02:48:58.549980 1732922 raft_consensus.cc:2468] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 04a5ea4feab74c3480f93fc3b9182d69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 407cabaabd22488a986388278be5c13f in term 1.
W20251027 02:48:58.550339 1732826 tablet.cc:2378] T 1036261eea4842f8b5274de21215f71e P 407cabaabd22488a986388278be5c13f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:48:58.550423 1733001 ts_tablet_manager.cc:1403] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20251027 02:48:58.550872 1732703 leader_election.cc:304] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a5ea4feab74c3480f93fc3b9182d69, 407cabaabd22488a986388278be5c13f; no voters:
I20251027 02:48:58.551326 1733002 tablet_bootstrap.cc:654] T e377623c494144c38bc40e664d53b097 P aa4328f7b8d74ea3aef5aa96cbf87961: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.551694 1733017 raft_consensus.cc:2804] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:48:58.551765 1732626 raft_consensus.cc:2468] T 43d4caa6e0d54e0391ade495eb2ac2f0 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 407cabaabd22488a986388278be5c13f in term 1.
I20251027 02:48:58.551923 1732627 raft_consensus.cc:2468] T dde9c836fd1a46ba83a3358453abbfb8 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 407cabaabd22488a986388278be5c13f in term 1.
I20251027 02:48:58.552268 1733004 tablet_bootstrap.cc:654] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69: Neither blocks nor log segments found. Creating new log.
I20251027 02:48:58.552173 1733017 raft_consensus.cc:697] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [term 1 LEADER]: Becoming Leader. State: Replica: 407cabaabd22488a986388278be5c13f, State: Running, Role: LEADER
W20251027 02:48:58.552614 1732857 leader_election.cc:343] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 407cabaabd22488a986388278be5c13f (127.152.231.194:45037): Illegal state: must be running to vote when last-logged opid is not known
I20251027 02:48:58.552987 1733001 raft_consensus.cc:359] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.553138 1732857 leader_election.cc:304] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 04a5ea4feab74c3480f93fc3b9182d69; no voters: 407cabaabd22488a986388278be5c13f, aa4328f7b8d74ea3aef5aa96cbf87961
I20251027 02:48:58.553336 1732704 leader_election.cc:304] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 407cabaabd22488a986388278be5c13f, aa4328f7b8d74ea3aef5aa96cbf87961; no voters:
I20251027 02:48:58.553066 1733017 consensus_queue.cc:237] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.553571 1732921 raft_consensus.cc:2468] T dde9c836fd1a46ba83a3358453abbfb8 P 04a5ea4feab74c3480f93fc3b9182d69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 407cabaabd22488a986388278be5c13f in term 1.
I20251027 02:48:58.553699 1733001 raft_consensus.cc:385] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.554000 1733007 raft_consensus.cc:2804] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:48:58.554164 1733010 raft_consensus.cc:2749] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251027 02:48:58.553994 1733001 raft_consensus.cc:740] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 407cabaabd22488a986388278be5c13f, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.554590 1733007 raft_consensus.cc:697] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [term 1 LEADER]: Becoming Leader. State: Replica: 407cabaabd22488a986388278be5c13f, State: Running, Role: LEADER
I20251027 02:48:58.554852 1733001 consensus_queue.cc:260] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.555403 1733007 consensus_queue.cc:237] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.556859 1733004 tablet_bootstrap.cc:492] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69: No bootstrap required, opened a new log
I20251027 02:48:58.557013 1733001 ts_tablet_manager.cc:1434] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20251027 02:48:58.557184 1733004 ts_tablet_manager.cc:1403] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20251027 02:48:58.558703 1733004 raft_consensus.cc:359] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.559114 1733004 raft_consensus.cc:385] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.559266 1733004 raft_consensus.cc:740] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a5ea4feab74c3480f93fc3b9182d69, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.559679 1733004 consensus_queue.cc:260] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.561301 1733004 ts_tablet_manager.cc:1434] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20251027 02:48:58.561810 1733002 tablet_bootstrap.cc:492] T e377623c494144c38bc40e664d53b097 P aa4328f7b8d74ea3aef5aa96cbf87961: No bootstrap required, opened a new log
I20251027 02:48:58.562258 1733002 ts_tablet_manager.cc:1403] T e377623c494144c38bc40e664d53b097 P aa4328f7b8d74ea3aef5aa96cbf87961: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.007s
I20251027 02:48:58.562790 1733008 raft_consensus.cc:493] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:48:58.563231 1733008 raft_consensus.cc:515] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.564742 1733002 raft_consensus.cc:359] T e377623c494144c38bc40e664d53b097 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.565393 1733002 raft_consensus.cc:385] T e377623c494144c38bc40e664d53b097 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:48:58.565636 1733002 raft_consensus.cc:740] T e377623c494144c38bc40e664d53b097 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa4328f7b8d74ea3aef5aa96cbf87961, State: Initialized, Role: FOLLOWER
I20251027 02:48:58.566375 1733002 consensus_queue.cc:260] T e377623c494144c38bc40e664d53b097 P aa4328f7b8d74ea3aef5aa96cbf87961 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.567755 1733008 leader_election.cc:290] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 407cabaabd22488a986388278be5c13f (127.152.231.194:45037), 04a5ea4feab74c3480f93fc3b9182d69 (127.152.231.195:46433)
I20251027 02:48:58.570459 1732242 catalog_manager.cc:5649] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f reported cstate change: term changed from 0 to 1, leader changed from <none> to 407cabaabd22488a986388278be5c13f (127.152.231.194). New cstate: current_term: 1 leader_uuid: "407cabaabd22488a986388278be5c13f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20251027 02:48:58.573175 1733002 ts_tablet_manager.cc:1434] T e377623c494144c38bc40e664d53b097 P aa4328f7b8d74ea3aef5aa96cbf87961: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
W20251027 02:48:58.580636 1732676 tablet.cc:2378] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:48:58.582618 1732774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a6ec10f95955403fac0d633b40dcc3f9" candidate_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "407cabaabd22488a986388278be5c13f" is_pre_election: true
I20251027 02:48:58.583294 1732774 raft_consensus.cc:2468] T a6ec10f95955403fac0d633b40dcc3f9 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa4328f7b8d74ea3aef5aa96cbf87961 in term 0.
I20251027 02:48:58.584451 1732561 leader_election.cc:304] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 407cabaabd22488a986388278be5c13f, aa4328f7b8d74ea3aef5aa96cbf87961; no voters:
I20251027 02:48:58.585358 1733038 raft_consensus.cc:2804] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:48:58.585641 1733038 raft_consensus.cc:493] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:48:58.585870 1733038 raft_consensus.cc:3060] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.586301 1732921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a6ec10f95955403fac0d633b40dcc3f9" candidate_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a5ea4feab74c3480f93fc3b9182d69" is_pre_election: true
I20251027 02:48:58.587008 1732921 raft_consensus.cc:2468] T a6ec10f95955403fac0d633b40dcc3f9 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa4328f7b8d74ea3aef5aa96cbf87961 in term 0.
I20251027 02:48:58.587260 1733010 raft_consensus.cc:493] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:48:58.587589 1733010 raft_consensus.cc:515] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.588987 1733010 leader_election.cc:290] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa4328f7b8d74ea3aef5aa96cbf87961 (127.152.231.193:34633), 407cabaabd22488a986388278be5c13f (127.152.231.194:45037)
I20251027 02:48:58.588655 1732242 catalog_manager.cc:5649] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f reported cstate change: term changed from 0 to 1, leader changed from <none> to 407cabaabd22488a986388278be5c13f (127.152.231.194). New cstate: current_term: 1 leader_uuid: "407cabaabd22488a986388278be5c13f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20251027 02:48:58.589852 1732627 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "643c36a0f0bc44309a0962ddc5c54437" candidate_uuid: "04a5ea4feab74c3480f93fc3b9182d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" is_pre_election: true
I20251027 02:48:58.589977 1732774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "643c36a0f0bc44309a0962ddc5c54437" candidate_uuid: "04a5ea4feab74c3480f93fc3b9182d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "407cabaabd22488a986388278be5c13f" is_pre_election: true
I20251027 02:48:58.590431 1732627 raft_consensus.cc:2468] T 643c36a0f0bc44309a0962ddc5c54437 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04a5ea4feab74c3480f93fc3b9182d69 in term 0.
I20251027 02:48:58.590574 1732774 raft_consensus.cc:2468] T 643c36a0f0bc44309a0962ddc5c54437 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04a5ea4feab74c3480f93fc3b9182d69 in term 0.
I20251027 02:48:58.591262 1732858 leader_election.cc:304] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a5ea4feab74c3480f93fc3b9182d69, aa4328f7b8d74ea3aef5aa96cbf87961; no voters:
I20251027 02:48:58.591216 1733038 raft_consensus.cc:515] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.591866 1733010 raft_consensus.cc:2804] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:48:58.592167 1733010 raft_consensus.cc:493] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:48:58.592377 1733010 raft_consensus.cc:3060] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.592406 1733038 leader_election.cc:290] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [CANDIDATE]: Term 1 election: Requested vote from peers 407cabaabd22488a986388278be5c13f (127.152.231.194:45037), 04a5ea4feab74c3480f93fc3b9182d69 (127.152.231.195:46433)
I20251027 02:48:58.593287 1732774 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a6ec10f95955403fac0d633b40dcc3f9" candidate_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "407cabaabd22488a986388278be5c13f"
I20251027 02:48:58.593542 1732921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a6ec10f95955403fac0d633b40dcc3f9" candidate_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a5ea4feab74c3480f93fc3b9182d69"
I20251027 02:48:58.593816 1732774 raft_consensus.cc:3060] T a6ec10f95955403fac0d633b40dcc3f9 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.593935 1732921 raft_consensus.cc:3060] T a6ec10f95955403fac0d633b40dcc3f9 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.596753 1733010 raft_consensus.cc:515] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.598194 1732921 raft_consensus.cc:2468] T a6ec10f95955403fac0d633b40dcc3f9 P 04a5ea4feab74c3480f93fc3b9182d69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa4328f7b8d74ea3aef5aa96cbf87961 in term 1.
I20251027 02:48:58.598222 1733010 leader_election.cc:290] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [CANDIDATE]: Term 1 election: Requested vote from peers aa4328f7b8d74ea3aef5aa96cbf87961 (127.152.231.193:34633), 407cabaabd22488a986388278be5c13f (127.152.231.194:45037)
I20251027 02:48:58.599040 1732627 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "643c36a0f0bc44309a0962ddc5c54437" candidate_uuid: "04a5ea4feab74c3480f93fc3b9182d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961"
I20251027 02:48:58.598999 1732561 leader_election.cc:304] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a5ea4feab74c3480f93fc3b9182d69, aa4328f7b8d74ea3aef5aa96cbf87961; no voters:
I20251027 02:48:58.599224 1732773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "643c36a0f0bc44309a0962ddc5c54437" candidate_uuid: "04a5ea4feab74c3480f93fc3b9182d69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "407cabaabd22488a986388278be5c13f"
I20251027 02:48:58.599613 1732627 raft_consensus.cc:3060] T 643c36a0f0bc44309a0962ddc5c54437 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.599715 1732773 raft_consensus.cc:3060] T 643c36a0f0bc44309a0962ddc5c54437 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.599820 1733038 raft_consensus.cc:2804] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:48:58.600157 1733038 raft_consensus.cc:697] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 1 LEADER]: Becoming Leader. State: Replica: aa4328f7b8d74ea3aef5aa96cbf87961, State: Running, Role: LEADER
I20251027 02:48:58.600394 1732774 raft_consensus.cc:2468] T a6ec10f95955403fac0d633b40dcc3f9 P 407cabaabd22488a986388278be5c13f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa4328f7b8d74ea3aef5aa96cbf87961 in term 1.
I20251027 02:48:58.600911 1733038 consensus_queue.cc:237] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.605032 1732773 raft_consensus.cc:2468] T 643c36a0f0bc44309a0962ddc5c54437 P 407cabaabd22488a986388278be5c13f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04a5ea4feab74c3480f93fc3b9182d69 in term 1.
I20251027 02:48:58.605516 1732627 raft_consensus.cc:2468] T 643c36a0f0bc44309a0962ddc5c54437 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04a5ea4feab74c3480f93fc3b9182d69 in term 1.
I20251027 02:48:58.605886 1732857 leader_election.cc:304] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a5ea4feab74c3480f93fc3b9182d69, 407cabaabd22488a986388278be5c13f; no voters:
I20251027 02:48:58.606467 1733010 raft_consensus.cc:2804] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:48:58.607301 1733038 raft_consensus.cc:493] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:48:58.607808 1733038 raft_consensus.cc:515] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.608481 1733010 raft_consensus.cc:697] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [term 1 LEADER]: Becoming Leader. State: Replica: 04a5ea4feab74c3480f93fc3b9182d69, State: Running, Role: LEADER
I20251027 02:48:58.609174 1733010 consensus_queue.cc:237] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.609771 1733038 leader_election.cc:290] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 407cabaabd22488a986388278be5c13f (127.152.231.194:45037), 04a5ea4feab74c3480f93fc3b9182d69 (127.152.231.195:46433)
I20251027 02:48:58.610553 1732773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1036261eea4842f8b5274de21215f71e" candidate_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "407cabaabd22488a986388278be5c13f" is_pre_election: true
I20251027 02:48:58.610910 1732921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1036261eea4842f8b5274de21215f71e" candidate_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a5ea4feab74c3480f93fc3b9182d69" is_pre_election: true
I20251027 02:48:58.611129 1732773 raft_consensus.cc:2468] T 1036261eea4842f8b5274de21215f71e P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa4328f7b8d74ea3aef5aa96cbf87961 in term 0.
I20251027 02:48:58.611486 1732921 raft_consensus.cc:2468] T 1036261eea4842f8b5274de21215f71e P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa4328f7b8d74ea3aef5aa96cbf87961 in term 0.
I20251027 02:48:58.612102 1732561 leader_election.cc:304] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 407cabaabd22488a986388278be5c13f, aa4328f7b8d74ea3aef5aa96cbf87961; no voters:
I20251027 02:48:58.612833 1733038 raft_consensus.cc:2804] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:48:58.613183 1733038 raft_consensus.cc:493] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:48:58.613466 1733038 raft_consensus.cc:3060] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.614017 1732242 catalog_manager.cc:5649] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa4328f7b8d74ea3aef5aa96cbf87961 (127.152.231.193). New cstate: current_term: 1 leader_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20251027 02:48:58.617635 1733038 raft_consensus.cc:515] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.618737 1733038 leader_election.cc:290] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [CANDIDATE]: Term 1 election: Requested vote from peers 407cabaabd22488a986388278be5c13f (127.152.231.194:45037), 04a5ea4feab74c3480f93fc3b9182d69 (127.152.231.195:46433)
I20251027 02:48:58.620353 1732773 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1036261eea4842f8b5274de21215f71e" candidate_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "407cabaabd22488a986388278be5c13f"
I20251027 02:48:58.620517 1732921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1036261eea4842f8b5274de21215f71e" candidate_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a5ea4feab74c3480f93fc3b9182d69"
I20251027 02:48:58.620951 1732921 raft_consensus.cc:3060] T 1036261eea4842f8b5274de21215f71e P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.620882 1732773 raft_consensus.cc:3060] T 1036261eea4842f8b5274de21215f71e P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.621061 1732241 catalog_manager.cc:5649] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04a5ea4feab74c3480f93fc3b9182d69 (127.152.231.195). New cstate: current_term: 1 leader_uuid: "04a5ea4feab74c3480f93fc3b9182d69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } health_report { overall_health: HEALTHY } } }
I20251027 02:48:58.625924 1732921 raft_consensus.cc:2468] T 1036261eea4842f8b5274de21215f71e P 04a5ea4feab74c3480f93fc3b9182d69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa4328f7b8d74ea3aef5aa96cbf87961 in term 1.
I20251027 02:48:58.626601 1732773 raft_consensus.cc:2468] T 1036261eea4842f8b5274de21215f71e P 407cabaabd22488a986388278be5c13f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa4328f7b8d74ea3aef5aa96cbf87961 in term 1.
I20251027 02:48:58.626729 1732561 leader_election.cc:304] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a5ea4feab74c3480f93fc3b9182d69, aa4328f7b8d74ea3aef5aa96cbf87961; no voters:
I20251027 02:48:58.627408 1733038 raft_consensus.cc:2804] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:48:58.627717 1733038 raft_consensus.cc:697] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [term 1 LEADER]: Becoming Leader. State: Replica: aa4328f7b8d74ea3aef5aa96cbf87961, State: Running, Role: LEADER
I20251027 02:48:58.628350 1733038 consensus_queue.cc:237] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.635742 1732241 catalog_manager.cc:5649] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa4328f7b8d74ea3aef5aa96cbf87961 (127.152.231.193). New cstate: current_term: 1 leader_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20251027 02:48:58.676060 1733007 raft_consensus.cc:493] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:48:58.676534 1733007 raft_consensus.cc:515] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.678385 1733007 leader_election.cc:290] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa4328f7b8d74ea3aef5aa96cbf87961 (127.152.231.193:34633), 04a5ea4feab74c3480f93fc3b9182d69 (127.152.231.195:46433)
I20251027 02:48:58.679176 1732627 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e377623c494144c38bc40e664d53b097" candidate_uuid: "407cabaabd22488a986388278be5c13f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" is_pre_election: true
I20251027 02:48:58.679328 1732921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e377623c494144c38bc40e664d53b097" candidate_uuid: "407cabaabd22488a986388278be5c13f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a5ea4feab74c3480f93fc3b9182d69" is_pre_election: true
I20251027 02:48:58.679733 1732921 raft_consensus.cc:2468] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 407cabaabd22488a986388278be5c13f in term 0.
I20251027 02:48:58.679772 1732627 raft_consensus.cc:2468] T e377623c494144c38bc40e664d53b097 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 407cabaabd22488a986388278be5c13f in term 0.
I20251027 02:48:58.680460 1732703 leader_election.cc:304] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a5ea4feab74c3480f93fc3b9182d69, 407cabaabd22488a986388278be5c13f; no voters:
I20251027 02:48:58.681098 1733007 raft_consensus.cc:2804] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:48:58.681408 1733007 raft_consensus.cc:493] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:48:58.681695 1733007 raft_consensus.cc:3060] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.687536 1733007 raft_consensus.cc:515] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
I20251027 02:48:58.689850 1733007 leader_election.cc:290] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [CANDIDATE]: Term 1 election: Requested vote from peers aa4328f7b8d74ea3aef5aa96cbf87961 (127.152.231.193:34633), 04a5ea4feab74c3480f93fc3b9182d69 (127.152.231.195:46433)
I20251027 02:48:58.690680 1732627 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e377623c494144c38bc40e664d53b097" candidate_uuid: "407cabaabd22488a986388278be5c13f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961"
I20251027 02:48:58.690789 1732921 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e377623c494144c38bc40e664d53b097" candidate_uuid: "407cabaabd22488a986388278be5c13f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a5ea4feab74c3480f93fc3b9182d69"
I20251027 02:48:58.691166 1732921 raft_consensus.cc:3060] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.691183 1732627 raft_consensus.cc:3060] T e377623c494144c38bc40e664d53b097 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:48:58.695113 1732921 raft_consensus.cc:2468] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 407cabaabd22488a986388278be5c13f in term 1.
I20251027 02:48:58.695950 1732703 leader_election.cc:304] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a5ea4feab74c3480f93fc3b9182d69, 407cabaabd22488a986388278be5c13f; no voters:
I20251027 02:48:58.696573 1733007 raft_consensus.cc:2804] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:48:58.696976 1733007 raft_consensus.cc:697] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [term 1 LEADER]: Becoming Leader. State: Replica: 407cabaabd22488a986388278be5c13f, State: Running, Role: LEADER
I20251027 02:48:58.697201 1732627 raft_consensus.cc:2468] T e377623c494144c38bc40e664d53b097 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 407cabaabd22488a986388278be5c13f in term 1.
I20251027 02:48:58.697659 1733007 consensus_queue.cc:237] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } }
W20251027 02:48:58.700098 1732971 tablet.cc:2378] T 1036261eea4842f8b5274de21215f71e P 04a5ea4feab74c3480f93fc3b9182d69: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:48:58.705864 1732241 catalog_manager.cc:5649] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f reported cstate change: term changed from 0 to 1, leader changed from <none> to 407cabaabd22488a986388278be5c13f (127.152.231.194). New cstate: current_term: 1 leader_uuid: "407cabaabd22488a986388278be5c13f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 } health_report { overall_health: UNKNOWN } } }
I20251027 02:48:58.947751 1732627 raft_consensus.cc:1275] T 643c36a0f0bc44309a0962ddc5c54437 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 1 FOLLOWER]: Refusing update from remote peer 04a5ea4feab74c3480f93fc3b9182d69: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:48:58.947752 1732773 raft_consensus.cc:1275] T 643c36a0f0bc44309a0962ddc5c54437 P 407cabaabd22488a986388278be5c13f [term 1 FOLLOWER]: Refusing update from remote peer 04a5ea4feab74c3480f93fc3b9182d69: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:48:58.949596 1733010 consensus_queue.cc:1048] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251027 02:48:58.950356 1732921 raft_consensus.cc:1275] T 1036261eea4842f8b5274de21215f71e P 04a5ea4feab74c3480f93fc3b9182d69 [term 1 FOLLOWER]: Refusing update from remote peer aa4328f7b8d74ea3aef5aa96cbf87961: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:48:58.950400 1732773 raft_consensus.cc:1275] T 1036261eea4842f8b5274de21215f71e P 407cabaabd22488a986388278be5c13f [term 1 FOLLOWER]: Refusing update from remote peer aa4328f7b8d74ea3aef5aa96cbf87961: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:48:58.950569 1733044 consensus_queue.cc:1048] T 643c36a0f0bc44309a0962ddc5c54437 P 04a5ea4feab74c3480f93fc3b9182d69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:48:58.950861 1732922 raft_consensus.cc:1275] T a6ec10f95955403fac0d633b40dcc3f9 P 04a5ea4feab74c3480f93fc3b9182d69 [term 1 FOLLOWER]: Refusing update from remote peer aa4328f7b8d74ea3aef5aa96cbf87961: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:48:58.951642 1732773 raft_consensus.cc:1275] T a6ec10f95955403fac0d633b40dcc3f9 P 407cabaabd22488a986388278be5c13f [term 1 FOLLOWER]: Refusing update from remote peer aa4328f7b8d74ea3aef5aa96cbf87961: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:48:58.951699 1733008 consensus_queue.cc:1048] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [LEADER]: Connected to new peer: Peer: permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:48:58.952291 1733038 consensus_queue.cc:1048] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:48:58.952419 1733066 consensus_queue.cc:1048] T 1036261eea4842f8b5274de21215f71e P aa4328f7b8d74ea3aef5aa96cbf87961 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251027 02:48:58.953305 1732627 raft_consensus.cc:1275] T dde9c836fd1a46ba83a3358453abbfb8 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 1 FOLLOWER]: Refusing update from remote peer 407cabaabd22488a986388278be5c13f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:48:58.953970 1732922 raft_consensus.cc:1275] T dde9c836fd1a46ba83a3358453abbfb8 P 04a5ea4feab74c3480f93fc3b9182d69 [term 1 FOLLOWER]: Refusing update from remote peer 407cabaabd22488a986388278be5c13f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:48:58.954602 1733038 consensus_queue.cc:1048] T a6ec10f95955403fac0d633b40dcc3f9 P aa4328f7b8d74ea3aef5aa96cbf87961 [LEADER]: Connected to new peer: Peer: permanent_uuid: "407cabaabd22488a986388278be5c13f" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 45037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:48:58.954977 1732921 raft_consensus.cc:1275] T e377623c494144c38bc40e664d53b097 P 04a5ea4feab74c3480f93fc3b9182d69 [term 1 FOLLOWER]: Refusing update from remote peer 407cabaabd22488a986388278be5c13f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:48:58.955404 1733007 consensus_queue.cc:1048] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [LEADER]: Connected to new peer: Peer: permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:48:58.957924 1733007 consensus_queue.cc:1048] T dde9c836fd1a46ba83a3358453abbfb8 P 407cabaabd22488a986388278be5c13f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:48:58.958446 1732922 raft_consensus.cc:1275] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 04a5ea4feab74c3480f93fc3b9182d69 [term 1 FOLLOWER]: Refusing update from remote peer 407cabaabd22488a986388278be5c13f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:48:58.959111 1732626 raft_consensus.cc:1275] T 43d4caa6e0d54e0391ade495eb2ac2f0 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 1 FOLLOWER]: Refusing update from remote peer 407cabaabd22488a986388278be5c13f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:48:58.960480 1733017 consensus_queue.cc:1048] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [LEADER]: Connected to new peer: Peer: permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:48:58.959760 1733068 consensus_queue.cc:1048] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [LEADER]: Connected to new peer: Peer: permanent_uuid: "04a5ea4feab74c3480f93fc3b9182d69" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251027 02:48:58.962985 1733017 consensus_queue.cc:1048] T 43d4caa6e0d54e0391ade495eb2ac2f0 P 407cabaabd22488a986388278be5c13f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:48:58.963636 1732626 raft_consensus.cc:1275] T e377623c494144c38bc40e664d53b097 P aa4328f7b8d74ea3aef5aa96cbf87961 [term 1 FOLLOWER]: Refusing update from remote peer 407cabaabd22488a986388278be5c13f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:48:58.966223 1733017 consensus_queue.cc:1048] T e377623c494144c38bc40e664d53b097 P 407cabaabd22488a986388278be5c13f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa4328f7b8d74ea3aef5aa96cbf87961" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:48:58.995011 1733063 mvcc.cc:204] Tried to move back new op lower bound from 7215240556333318144 to 7215240554722832384. Current Snapshot: MvccSnapshot[applied={T|T < 7215240556333318144}]
I20251027 02:48:59.022800 1733074 mvcc.cc:204] Tried to move back new op lower bound from 7215240556332855296 to 7215240554730881024. Current Snapshot: MvccSnapshot[applied={T|T < 7215240556332855296}]
I20251027 02:48:59.028460 1733062 mvcc.cc:204] Tried to move back new op lower bound from 7215240556333789184 to 7215240555312525312. Current Snapshot: MvccSnapshot[applied={T|T < 7215240556333789184}]
W20251027 02:49:01.542757 1733127 debug-util.cc:398] Leaking SignalData structure 0x7b08000aec40 after lost signal to thread 1732075
W20251027 02:49:01.543308 1733127 debug-util.cc:398] Leaking SignalData structure 0x7b08000962c0 after lost signal to thread 1732277
W20251027 02:49:06.494892 1733181 debug-util.cc:398] Leaking SignalData structure 0x7b08000a4d00 after lost signal to thread 1732295
W20251027 02:49:06.495843 1733181 debug-util.cc:398] Leaking SignalData structure 0x7b08000a1960 after lost signal to thread 1732369
W20251027 02:49:11.179782 1733230 debug-util.cc:398] Leaking SignalData structure 0x7b08000980c0 after lost signal to thread 1732380
W20251027 02:49:11.180423 1733230 debug-util.cc:398] Leaking SignalData structure 0x7b080009b880 after lost signal to thread 1732467
W20251027 02:49:15.761943 1732818 debug-util.cc:398] Leaking SignalData structure 0x7b08000b7560 after lost signal to thread 1732684
W20251027 02:49:15.762918 1732818 debug-util.cc:398] Leaking SignalData structure 0x7b08000e8880 after lost signal to thread 1732821
W20251027 02:49:16.950162 1733291 debug-util.cc:398] Leaking SignalData structure 0x7b08000810c0 after lost signal to thread 1732514
W20251027 02:49:16.951493 1733291 debug-util.cc:398] Leaking SignalData structure 0x7b08000b7520 after lost signal to thread 1732671
W20251027 02:49:20.555406 1732366 debug-util.cc:398] Leaking SignalData structure 0x7b080009eb20 after lost signal to thread 1732295
W20251027 02:49:20.556201 1732366 debug-util.cc:398] Leaking SignalData structure 0x7b08000a67a0 after lost signal to thread 1732369
W20251027 02:49:22.982143 1733373 debug-util.cc:398] Leaking SignalData structure 0x7b08000bd120 after lost signal to thread 1732684
W20251027 02:49:22.983357 1733373 debug-util.cc:398] Leaking SignalData structure 0x7b08000e8d40 after lost signal to thread 1732821
W20251027 02:49:28.989501 1733395 debug-util.cc:398] Leaking SignalData structure 0x7b08000d3300 after lost signal to thread 1732838
W20251027 02:49:28.990800 1733395 debug-util.cc:398] Leaking SignalData structure 0x7b08000bfb00 after lost signal to thread 1732966
I20251027 02:49:33.385655 1733291 scanners.cc:360] slow scan reporting is disabled: set --show_slow_scans to enable
W20251027 02:49:33.395704 1732668 debug-util.cc:398] Leaking SignalData structure 0x7b08000b5140 after lost signal to thread 1732514
W20251027 02:49:33.396695 1732668 debug-util.cc:398] Leaking SignalData structure 0x7b08000553c0 after lost signal to thread 1732671
I20251027 02:49:34.334232 1733373 scanners.cc:360] slow scan reporting is disabled: set --show_slow_scans to enable
I20251027 02:49:35.804404 1733395 scanners.cc:360] slow scan reporting is disabled: set --show_slow_scans to enable
I20251027 02:49:38.863330 1729439 webserver-crawl-itest.cc:326] Dumping visited URLs
I20251027 02:49:38.863662 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903
I20251027 02:49:38.863849 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/bootstrap/js/bootstrap-table.min.js
I20251027 02:49:38.864012 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/bootstrap/js/bootstrap.min.js
I20251027 02:49:38.864167 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/config
I20251027 02:49:38.864331 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/dashboards
I20251027 02:49:38.864485 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/healthz
I20251027 02:49:38.864631 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/jquery-3.5.1.min.js
I20251027 02:49:38.864777 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/kudu.js
I20251027 02:49:38.864938 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/log-anchors?id=1036261eea4842f8b5274de21215f71e
I20251027 02:49:38.865100 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/log-anchors?id=43d4caa6e0d54e0391ade495eb2ac2f0
I20251027 02:49:38.865255 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/log-anchors?id=643c36a0f0bc44309a0962ddc5c54437
I20251027 02:49:38.865401 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/log-anchors?id=a6ec10f95955403fac0d633b40dcc3f9
I20251027 02:49:38.865538 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/log-anchors?id=dde9c836fd1a46ba83a3358453abbfb8
I20251027 02:49:38.865679 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/log-anchors?id=e377623c494144c38bc40e664d53b097
I20251027 02:49:38.865832 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/logs
I20251027 02:49:38.865974 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/maintenance-manager
I20251027 02:49:38.866127 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/mem-trackers
I20251027 02:49:38.866288 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/memz
I20251027 02:49:38.866436 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/metrics
I20251027 02:49:38.866583 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/metrics_prometheus
I20251027 02:49:38.866734 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/rpcz
I20251027 02:49:38.866878 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/scans
I20251027 02:49:38.867029 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/stacks
I20251027 02:49:38.867184 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/startup
I20251027 02:49:38.867332 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet-consensus-status?id=1036261eea4842f8b5274de21215f71e
I20251027 02:49:38.867475 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet-consensus-status?id=43d4caa6e0d54e0391ade495eb2ac2f0
I20251027 02:49:38.867616 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet-consensus-status?id=643c36a0f0bc44309a0962ddc5c54437
I20251027 02:49:38.867764 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet-consensus-status?id=a6ec10f95955403fac0d633b40dcc3f9
I20251027 02:49:38.867906 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet-consensus-status?id=dde9c836fd1a46ba83a3358453abbfb8
I20251027 02:49:38.868057 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet-consensus-status?id=e377623c494144c38bc40e664d53b097
I20251027 02:49:38.868216 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet-rowsetlayout-svg?id=1036261eea4842f8b5274de21215f71e
I20251027 02:49:38.868363 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet-rowsetlayout-svg?id=43d4caa6e0d54e0391ade495eb2ac2f0
I20251027 02:49:38.868502 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet-rowsetlayout-svg?id=643c36a0f0bc44309a0962ddc5c54437
I20251027 02:49:38.868644 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet-rowsetlayout-svg?id=a6ec10f95955403fac0d633b40dcc3f9
I20251027 02:49:38.868782 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet-rowsetlayout-svg?id=dde9c836fd1a46ba83a3358453abbfb8
I20251027 02:49:38.868922 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet-rowsetlayout-svg?id=e377623c494144c38bc40e664d53b097
I20251027 02:49:38.869068 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet?id=1036261eea4842f8b5274de21215f71e
I20251027 02:49:38.869248 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet?id=43d4caa6e0d54e0391ade495eb2ac2f0
I20251027 02:49:38.869395 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet?id=643c36a0f0bc44309a0962ddc5c54437
I20251027 02:49:38.869540 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet?id=a6ec10f95955403fac0d633b40dcc3f9
I20251027 02:49:38.869681 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet?id=dde9c836fd1a46ba83a3358453abbfb8
I20251027 02:49:38.869818 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablet?id=e377623c494144c38bc40e664d53b097
I20251027 02:49:38.869973 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/tablets
I20251027 02:49:38.870123 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz
I20251027 02:49:38.870273 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=acceptor%20pool
I20251027 02:49:38.870405 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=all
I20251027 02:49:38.870530 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=cache
I20251027 02:49:38.870671 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=cloud%20detector
I20251027 02:49:38.870819 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=heartbeater
I20251027 02:49:38.870961 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=kernel-watchdog
I20251027 02:49:38.871104 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=maintenance
I20251027 02:49:38.871258 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=ntp
I20251027 02:49:38.871397 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=reactor
I20251027 02:49:38.871543 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=scanners
I20251027 02:49:38.871686 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=server
I20251027 02:49:38.871832 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:49:38.871975 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:49:38.872115 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:49:38.872269 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=service%20pool%20kudu.tserver.TabletServerAdminService
I20251027 02:49:38.872413 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=service%20pool%20kudu.tserver.TabletServerService
I20251027 02:49:38.872558 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=tablet-copy
I20251027 02:49:38.872706 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/threadz?group=thread%20pool
I20251027 02:49:38.872854 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/transactions
I20251027 02:49:38.873000 1729439 webserver-crawl-itest.cc:328] http://127.152.231.193:32903/varz
I20251027 02:49:38.873175 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827
I20251027 02:49:38.873337 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/bootstrap/js/bootstrap-table.min.js
I20251027 02:49:38.873479 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/bootstrap/js/bootstrap.min.js
I20251027 02:49:38.873622 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/config
I20251027 02:49:38.873759 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/dashboards
I20251027 02:49:38.873891 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/healthz
I20251027 02:49:38.874042 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/jquery-3.5.1.min.js
I20251027 02:49:38.874179 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/kudu.js
I20251027 02:49:38.874334 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/log-anchors?id=1036261eea4842f8b5274de21215f71e
I20251027 02:49:38.874481 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/log-anchors?id=43d4caa6e0d54e0391ade495eb2ac2f0
I20251027 02:49:38.874627 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/log-anchors?id=643c36a0f0bc44309a0962ddc5c54437
I20251027 02:49:38.874771 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/log-anchors?id=a6ec10f95955403fac0d633b40dcc3f9
I20251027 02:49:38.874912 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/log-anchors?id=dde9c836fd1a46ba83a3358453abbfb8
I20251027 02:49:38.875054 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/log-anchors?id=e377623c494144c38bc40e664d53b097
I20251027 02:49:38.875200 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/logs
I20251027 02:49:38.875355 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/maintenance-manager
I20251027 02:49:38.875505 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/mem-trackers
I20251027 02:49:38.875655 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/memz
I20251027 02:49:38.875804 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/metrics
I20251027 02:49:38.875950 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/metrics_prometheus
I20251027 02:49:38.876091 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/rpcz
I20251027 02:49:38.876246 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/scans
I20251027 02:49:38.876398 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/stacks
I20251027 02:49:38.876554 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/startup
I20251027 02:49:38.876701 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet-consensus-status?id=1036261eea4842f8b5274de21215f71e
I20251027 02:49:38.876848 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet-consensus-status?id=43d4caa6e0d54e0391ade495eb2ac2f0
I20251027 02:49:38.876993 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet-consensus-status?id=643c36a0f0bc44309a0962ddc5c54437
I20251027 02:49:38.877156 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet-consensus-status?id=a6ec10f95955403fac0d633b40dcc3f9
I20251027 02:49:38.877318 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet-consensus-status?id=dde9c836fd1a46ba83a3358453abbfb8
I20251027 02:49:38.877465 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet-consensus-status?id=e377623c494144c38bc40e664d53b097
I20251027 02:49:38.877604 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet-rowsetlayout-svg?id=1036261eea4842f8b5274de21215f71e
I20251027 02:49:38.877744 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet-rowsetlayout-svg?id=43d4caa6e0d54e0391ade495eb2ac2f0
I20251027 02:49:38.877912 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet-rowsetlayout-svg?id=643c36a0f0bc44309a0962ddc5c54437
I20251027 02:49:38.878057 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet-rowsetlayout-svg?id=a6ec10f95955403fac0d633b40dcc3f9
I20251027 02:49:38.878201 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet-rowsetlayout-svg?id=dde9c836fd1a46ba83a3358453abbfb8
I20251027 02:49:38.878353 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet-rowsetlayout-svg?id=e377623c494144c38bc40e664d53b097
I20251027 02:49:38.878504 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet?id=1036261eea4842f8b5274de21215f71e
I20251027 02:49:38.878659 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet?id=43d4caa6e0d54e0391ade495eb2ac2f0
I20251027 02:49:38.878805 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet?id=643c36a0f0bc44309a0962ddc5c54437
I20251027 02:49:38.878949 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet?id=a6ec10f95955403fac0d633b40dcc3f9
I20251027 02:49:38.879096 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet?id=dde9c836fd1a46ba83a3358453abbfb8
I20251027 02:49:38.879247 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablet?id=e377623c494144c38bc40e664d53b097
I20251027 02:49:38.879398 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/tablets
I20251027 02:49:38.879546 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz
I20251027 02:49:38.879699 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=acceptor%20pool
I20251027 02:49:38.879844 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=all
I20251027 02:49:38.879988 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=cache
I20251027 02:49:38.880139 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=cloud%20detector
I20251027 02:49:38.880300 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=heartbeater
I20251027 02:49:38.880451 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=kernel-watchdog
I20251027 02:49:38.880604 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=maintenance
I20251027 02:49:38.880756 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=ntp
I20251027 02:49:38.880909 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=reactor
I20251027 02:49:38.881062 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=scanners
I20251027 02:49:38.881246 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=server
I20251027 02:49:38.881400 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:49:38.881556 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:49:38.881705 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:49:38.881850 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=service%20pool%20kudu.tserver.TabletServerAdminService
I20251027 02:49:38.881989 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=service%20pool%20kudu.tserver.TabletServerService
I20251027 02:49:38.882138 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=tablet-copy
I20251027 02:49:38.882294 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/threadz?group=thread%20pool
I20251027 02:49:38.882447 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/transactions
I20251027 02:49:38.882593 1729439 webserver-crawl-itest.cc:328] http://127.152.231.194:46827/varz
I20251027 02:49:38.882746 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309
I20251027 02:49:38.882899 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/bootstrap/js/bootstrap-table.min.js
I20251027 02:49:38.883060 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/bootstrap/js/bootstrap.min.js
I20251027 02:49:38.883199 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/config
I20251027 02:49:38.883371 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/dashboards
I20251027 02:49:38.883518 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/healthz
I20251027 02:49:38.883672 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/jquery-3.5.1.min.js
I20251027 02:49:38.883819 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/kudu.js
I20251027 02:49:38.883967 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/log-anchors?id=1036261eea4842f8b5274de21215f71e
I20251027 02:49:38.884125 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/log-anchors?id=43d4caa6e0d54e0391ade495eb2ac2f0
I20251027 02:49:38.884284 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/log-anchors?id=643c36a0f0bc44309a0962ddc5c54437
I20251027 02:49:38.884434 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/log-anchors?id=a6ec10f95955403fac0d633b40dcc3f9
I20251027 02:49:38.884579 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/log-anchors?id=dde9c836fd1a46ba83a3358453abbfb8
I20251027 02:49:38.884721 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/log-anchors?id=e377623c494144c38bc40e664d53b097
I20251027 02:49:38.884866 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/logs
I20251027 02:49:38.885008 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/maintenance-manager
I20251027 02:49:38.885185 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/mem-trackers
I20251027 02:49:38.885347 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/memz
I20251027 02:49:38.885501 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/metrics
I20251027 02:49:38.885651 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/metrics_prometheus
I20251027 02:49:38.885805 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/rpcz
I20251027 02:49:38.885952 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/scans
I20251027 02:49:38.886102 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/stacks
I20251027 02:49:38.886261 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/startup
I20251027 02:49:38.886413 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet-consensus-status?id=1036261eea4842f8b5274de21215f71e
I20251027 02:49:38.886564 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet-consensus-status?id=43d4caa6e0d54e0391ade495eb2ac2f0
I20251027 02:49:38.886716 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet-consensus-status?id=643c36a0f0bc44309a0962ddc5c54437
I20251027 02:49:38.886868 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet-consensus-status?id=a6ec10f95955403fac0d633b40dcc3f9
I20251027 02:49:38.887017 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet-consensus-status?id=dde9c836fd1a46ba83a3358453abbfb8
I20251027 02:49:38.887167 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet-consensus-status?id=e377623c494144c38bc40e664d53b097
I20251027 02:49:38.887322 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet-rowsetlayout-svg?id=1036261eea4842f8b5274de21215f71e
I20251027 02:49:38.887465 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet-rowsetlayout-svg?id=43d4caa6e0d54e0391ade495eb2ac2f0
I20251027 02:49:38.887611 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet-rowsetlayout-svg?id=643c36a0f0bc44309a0962ddc5c54437
I20251027 02:49:38.887760 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet-rowsetlayout-svg?id=a6ec10f95955403fac0d633b40dcc3f9
I20251027 02:49:38.887907 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet-rowsetlayout-svg?id=dde9c836fd1a46ba83a3358453abbfb8
I20251027 02:49:38.888051 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet-rowsetlayout-svg?id=e377623c494144c38bc40e664d53b097
I20251027 02:49:38.888197 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet?id=1036261eea4842f8b5274de21215f71e
I20251027 02:49:38.888361 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet?id=43d4caa6e0d54e0391ade495eb2ac2f0
I20251027 02:49:38.888512 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet?id=643c36a0f0bc44309a0962ddc5c54437
I20251027 02:49:38.888670 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet?id=a6ec10f95955403fac0d633b40dcc3f9
I20251027 02:49:38.888814 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet?id=dde9c836fd1a46ba83a3358453abbfb8
I20251027 02:49:38.888957 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablet?id=e377623c494144c38bc40e664d53b097
I20251027 02:49:38.889123 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/tablets
I20251027 02:49:38.889286 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz
I20251027 02:49:38.889437 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=acceptor%20pool
I20251027 02:49:38.889585 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=all
I20251027 02:49:38.889732 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=cache
I20251027 02:49:38.889883 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=cloud%20detector
I20251027 02:49:38.890034 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=heartbeater
I20251027 02:49:38.890182 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=kernel-watchdog
I20251027 02:49:38.890345 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=maintenance
I20251027 02:49:38.890501 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=ntp
I20251027 02:49:38.890643 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=reactor
I20251027 02:49:38.890790 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=scanners
I20251027 02:49:38.890935 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=server
I20251027 02:49:38.891080 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:49:38.891238 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:49:38.891387 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:49:38.891525 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=service%20pool%20kudu.tserver.TabletServerAdminService
I20251027 02:49:38.891667 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=service%20pool%20kudu.tserver.TabletServerService
I20251027 02:49:38.891809 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=tablet-copy
I20251027 02:49:38.891942 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/threadz?group=thread%20pool
I20251027 02:49:38.892073 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/transactions
I20251027 02:49:38.892211 1729439 webserver-crawl-itest.cc:328] http://127.152.231.195:46309/varz
I20251027 02:49:38.892362 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417
I20251027 02:49:38.892508 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/bootstrap/js/bootstrap-table.min.js
I20251027 02:49:38.892656 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/bootstrap/js/bootstrap.min.js
I20251027 02:49:38.892807 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/config
I20251027 02:49:38.892957 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/healthz
I20251027 02:49:38.893123 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/ipki-ca-cert
I20251027 02:49:38.893287 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/jquery-3.5.1.min.js
I20251027 02:49:38.893435 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/kudu.js
I20251027 02:49:38.893594 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/logs
I20251027 02:49:38.893744 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/masters
I20251027 02:49:38.893893 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/mem-trackers
I20251027 02:49:38.894040 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/memz
I20251027 02:49:38.894186 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/metrics
I20251027 02:49:38.894345 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/metrics_prometheus
I20251027 02:49:38.894495 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/rpcz
I20251027 02:49:38.894647 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/stacks
I20251027 02:49:38.894791 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/startup
I20251027 02:49:38.894935 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/tables
I20251027 02:49:38.895085 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/tablet-servers
I20251027 02:49:38.895242 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz
I20251027 02:49:38.895394 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz?group=acceptor%20pool
I20251027 02:49:38.895540 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz?group=all
I20251027 02:49:38.895689 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz?group=cache
I20251027 02:49:38.895838 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz?group=catalog%20manager
I20251027 02:49:38.895987 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz?group=cloud%20detector
I20251027 02:49:38.896136 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz?group=kernel-watchdog
I20251027 02:49:38.896294 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz?group=maintenance
I20251027 02:49:38.896443 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz?group=master
I20251027 02:49:38.896620 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz?group=ntp
I20251027 02:49:38.896766 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz?group=reactor
I20251027 02:49:38.896911 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz?group=server
I20251027 02:49:38.897059 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:49:38.897239 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz?group=service%20pool%20kudu.master.MasterService
I20251027 02:49:38.897394 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:49:38.897541 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:49:38.897696 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz?group=tablet-copy
I20251027 02:49:38.897876 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/threadz?group=thread%20pool
I20251027 02:49:38.898023 1729439 webserver-crawl-itest.cc:328] http://127.152.231.252:44417/varz
I20251027 02:49:38.898171 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229
I20251027 02:49:38.898329 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/bootstrap/js/bootstrap-table.min.js
I20251027 02:49:38.898478 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/bootstrap/js/bootstrap.min.js
I20251027 02:49:38.898627 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/config
I20251027 02:49:38.898778 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/healthz
I20251027 02:49:38.898924 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/ipki-ca-cert
I20251027 02:49:38.899073 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/jquery-3.5.1.min.js
I20251027 02:49:38.899222 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/kudu.js
I20251027 02:49:38.899379 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/logs
I20251027 02:49:38.899528 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/masters
I20251027 02:49:38.899688 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/mem-trackers
I20251027 02:49:38.899829 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/memz
I20251027 02:49:38.899994 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/metrics
I20251027 02:49:38.900152 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/metrics_prometheus
I20251027 02:49:38.900321 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/rpcz
I20251027 02:49:38.900476 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/stacks
I20251027 02:49:38.900624 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/startup
I20251027 02:49:38.900772 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/tables
I20251027 02:49:38.900926 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/tablet-servers
I20251027 02:49:38.901108 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz
I20251027 02:49:38.901269 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz?group=acceptor%20pool
I20251027 02:49:38.901419 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz?group=all
I20251027 02:49:38.901568 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz?group=cache
I20251027 02:49:38.901726 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz?group=catalog%20manager
I20251027 02:49:38.901872 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz?group=cloud%20detector
I20251027 02:49:38.902019 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz?group=kernel-watchdog
I20251027 02:49:38.902168 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz?group=maintenance
I20251027 02:49:38.902324 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz?group=master
I20251027 02:49:38.902473 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz?group=ntp
I20251027 02:49:38.902619 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz?group=reactor
I20251027 02:49:38.902765 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz?group=server
I20251027 02:49:38.902910 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:49:38.903064 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz?group=service%20pool%20kudu.master.MasterService
I20251027 02:49:38.903220 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:49:38.903378 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:49:38.903525 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz?group=tablet-copy
I20251027 02:49:38.903682 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/threadz?group=thread%20pool
I20251027 02:49:38.903825 1729439 webserver-crawl-itest.cc:328] http://127.152.231.253:41229/varz
I20251027 02:49:38.903967 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271
I20251027 02:49:38.904112 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/bootstrap/js/bootstrap-table.min.js
I20251027 02:49:38.904269 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/bootstrap/js/bootstrap.min.js
I20251027 02:49:38.904417 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/config
I20251027 02:49:38.904563 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/healthz
I20251027 02:49:38.904704 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/ipki-ca-cert
I20251027 02:49:38.904850 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/jquery-3.5.1.min.js
I20251027 02:49:38.904999 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/kudu.js
I20251027 02:49:38.905169 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/logs
I20251027 02:49:38.905329 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/masters
I20251027 02:49:38.905473 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/mem-trackers
I20251027 02:49:38.905617 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/memz
I20251027 02:49:38.905761 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/metrics
I20251027 02:49:38.905894 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/metrics_prometheus
I20251027 02:49:38.906016 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/rpcz
I20251027 02:49:38.906145 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/stacks
I20251027 02:49:38.906304 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/startup
I20251027 02:49:38.906447 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/table?id=844ef6d9458447798a2dfb2a6088af3f
I20251027 02:49:38.906592 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/tables
I20251027 02:49:38.906735 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/tables?&redirects=1
I20251027 02:49:38.906883 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/tablet-servers
I20251027 02:49:38.907027 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz
I20251027 02:49:38.907176 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz?group=acceptor%20pool
I20251027 02:49:38.907335 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz?group=all
I20251027 02:49:38.907480 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz?group=cache
I20251027 02:49:38.907616 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz?group=catalog%20manager
I20251027 02:49:38.907749 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz?group=cloud%20detector
I20251027 02:49:38.907894 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz?group=kernel-watchdog
I20251027 02:49:38.908044 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz?group=maintenance
I20251027 02:49:38.908196 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz?group=master
I20251027 02:49:38.908351 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz?group=ntp
I20251027 02:49:38.908499 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz?group=reactor
I20251027 02:49:38.908646 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz?group=server
I20251027 02:49:38.908790 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:49:38.908931 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz?group=service%20pool%20kudu.master.MasterService
I20251027 02:49:38.909091 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:49:38.909250 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:49:38.909399 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz?group=tablet-copy
I20251027 02:49:38.909541 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/threadz?group=thread%20pool
I20251027 02:49:38.909687 1729439 webserver-crawl-itest.cc:328] http://127.152.231.254:45271/varz
I20251027 02:49:38.913537 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1732513
I20251027 02:49:38.971877 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1732682
W20251027 02:49:38.971836 1732704 connection.cc:537] client connection to 127.152.231.193:34633 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251027 02:49:39.030398 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1732832
I20251027 02:49:39.085000 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1732069
I20251027 02:49:39.136359 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1732280
I20251027 02:49:39.178930 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1732372
2025-10-27T02:49:39Z chronyd exiting
[ OK ] WebserverCrawlITest.TestAllWebPages/1 (55492 ms)
[ RUN ] WebserverCrawlITest.TestAllWebPages/2
I20251027 02:49:39.279736 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
2025-10-27T02:49:39Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-10-27T02:49:39Z Disabled control of system clock
I20251027 02:49:39.331063 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:49:39.331581 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:49:39.332005 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:49:39.334678 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.254:38201
--webserver_interface=127.152.231.254
--webserver_port=0
--builtin_ntp_servers=127.152.231.212:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.152.231.254:38201,127.152.231.253:44983,127.152.231.252:36319
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.cert
--webserver_private_key_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.key
--webserver_private_key_password_cmd=echo test with env {}
W20251027 02:49:39.875150 1733604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:49:39.876022 1733604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:49:39.877039 1733604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:49:39.938581 1733604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251027 02:49:39.939059 1733604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:49:39.939425 1733604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251027 02:49:39.939783 1733604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251027 02:49:40.008908 1733604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.152.231.254:38201,127.152.231.253:44983,127.152.231.252:36319
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.254:38201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-0/data/info.pb
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.cert
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.254
--webserver_port=0
--webserver_private_key_file=<redacted>
--webserver_private_key_password_cmd=<redacted>
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:49:40.010898 1733604 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:49:40.013500 1733604 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:49:40.031865 1733610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:49:40.033442 1733611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:49:41.392072 1733613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:49:41.394928 1733612 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1354 milliseconds
W20251027 02:49:41.396129 1733604 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.364s user 0.599s sys 0.763s
W20251027 02:49:41.396522 1733604 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.364s user 0.599s sys 0.763s
I20251027 02:49:41.396813 1733604 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:49:41.398466 1733604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:49:41.402882 1733604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:49:41.404407 1733604 hybrid_clock.cc:648] HybridClock initialized: now 1761533381404332 us; error 53 us; skew 500 ppm
I20251027 02:49:41.405716 1733604 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:49:41.442919 1733604 webserver.cc:492] Webserver started at https://127.152.231.254:40259/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:49:41.444330 1733604 fs_manager.cc:362] Metadata directory not provided
I20251027 02:49:41.444586 1733604 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:49:41.445250 1733604 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:49:41.451728 1733604 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-0/data/instance:
uuid: "754fb6432e6f4fc89bfe76c08015f9c3"
format_stamp: "Formatted at 2025-10-27 02:49:41 on 5fd53c4cbb9d"
I20251027 02:49:41.452903 1733604 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-0/wal/instance:
uuid: "754fb6432e6f4fc89bfe76c08015f9c3"
format_stamp: "Formatted at 2025-10-27 02:49:41 on 5fd53c4cbb9d"
I20251027 02:49:41.462437 1733604 fs_manager.cc:696] Time spent creating directory manager: real 0.009s user 0.006s sys 0.002s
I20251027 02:49:41.469831 1733637 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:49:41.471166 1733604 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20251027 02:49:41.471493 1733604 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-0/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-0/wal
uuid: "754fb6432e6f4fc89bfe76c08015f9c3"
format_stamp: "Formatted at 2025-10-27 02:49:41 on 5fd53c4cbb9d"
I20251027 02:49:41.471835 1733604 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-0/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-0/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:49:41.558329 1733604 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:49:41.559679 1733604 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:49:41.560125 1733604 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:49:41.656294 1733604 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.254:38201
I20251027 02:49:41.656363 1733688 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.254:38201 every 8 connection(s)
I20251027 02:49:41.659271 1733604 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-0/data/info.pb
I20251027 02:49:41.666183 1733689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:41.667138 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1733604
I20251027 02:49:41.667688 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-0/wal/instance
I20251027 02:49:41.671501 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.253:44983
--webserver_interface=127.152.231.253
--webserver_port=0
--builtin_ntp_servers=127.152.231.212:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.152.231.254:38201,127.152.231.253:44983,127.152.231.252:36319
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.cert
--webserver_private_key_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.key
--webserver_private_key_password_cmd=echo test with env {}
I20251027 02:49:41.678193 1733689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:49:41.707459 1733689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } has no permanent_uuid. Determining permanent_uuid...
W20251027 02:49:41.709954 1733640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.152.231.253:44983: connect: Connection refused (error 111)
W20251027 02:49:41.715065 1733689 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:44983: Network error: Client connection negotiation failed: client connection to 127.152.231.253:44983: connect: Connection refused (error 111)
I20251027 02:49:41.764545 1733689 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } attempt: 1
W20251027 02:49:41.768419 1733689 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:44983: Network error: Client connection negotiation failed: client connection to 127.152.231.253:44983: connect: Connection refused (error 111)
I20251027 02:49:41.836786 1733689 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } attempt: 2
W20251027 02:49:41.840646 1733689 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:44983: Network error: Client connection negotiation failed: client connection to 127.152.231.253:44983: connect: Connection refused (error 111)
I20251027 02:49:41.932042 1733689 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } attempt: 3
W20251027 02:49:41.935995 1733689 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:44983: Network error: Client connection negotiation failed: client connection to 127.152.231.253:44983: connect: Connection refused (error 111)
W20251027 02:49:42.027329 1733691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:49:42.027910 1733691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:49:42.028457 1733691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:49:42.061266 1733691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251027 02:49:42.061551 1733691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:49:42.061748 1733691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251027 02:49:42.061937 1733691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251027 02:49:42.079377 1733689 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } attempt: 4
W20251027 02:49:42.083850 1733689 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:44983: Network error: Client connection negotiation failed: client connection to 127.152.231.253:44983: connect: Connection refused (error 111)
I20251027 02:49:42.099797 1733691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.152.231.254:38201,127.152.231.253:44983,127.152.231.252:36319
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.253:44983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-1/data/info.pb
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.cert
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.253
--webserver_port=0
--webserver_private_key_file=<redacted>
--webserver_private_key_password_cmd=<redacted>
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:49:42.101207 1733691 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:49:42.102913 1733691 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:49:42.117916 1733702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:49:42.383247 1733689 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } attempt: 5
W20251027 02:49:42.388509 1733689 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:44983: Network error: Client connection negotiation failed: client connection to 127.152.231.253:44983: connect: Connection refused (error 111)
I20251027 02:49:42.936048 1733689 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } attempt: 6
W20251027 02:49:42.943797 1733689 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:44983: Network error: Client connection negotiation failed: client connection to 127.152.231.253:44983: connect: Connection refused (error 111)
W20251027 02:49:42.117964 1733703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:49:43.289263 1733691 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.171s user 0.462s sys 0.709s
W20251027 02:49:43.289602 1733691 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.172s user 0.462s sys 0.709s
W20251027 02:49:43.289867 1733704 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1165 milliseconds
W20251027 02:49:43.290354 1733705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:49:43.290393 1733691 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:49:43.291586 1733691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:49:43.295022 1733691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:49:43.296360 1733691 hybrid_clock.cc:648] HybridClock initialized: now 1761533383296311 us; error 42 us; skew 500 ppm
I20251027 02:49:43.297212 1733691 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:49:43.326018 1733691 webserver.cc:492] Webserver started at https://127.152.231.253:40897/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:49:43.327258 1733691 fs_manager.cc:362] Metadata directory not provided
I20251027 02:49:43.327481 1733691 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:49:43.327979 1733691 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:49:43.334090 1733691 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-1/data/instance:
uuid: "5d6560dc68c84e38898fe92d1d47e8a9"
format_stamp: "Formatted at 2025-10-27 02:49:43 on 5fd53c4cbb9d"
I20251027 02:49:43.335170 1733691 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-1/wal/instance:
uuid: "5d6560dc68c84e38898fe92d1d47e8a9"
format_stamp: "Formatted at 2025-10-27 02:49:43 on 5fd53c4cbb9d"
I20251027 02:49:43.343182 1733691 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.000s sys 0.008s
I20251027 02:49:43.348868 1733731 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:49:43.350083 1733691 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20251027 02:49:43.350359 1733691 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-1/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-1/wal
uuid: "5d6560dc68c84e38898fe92d1d47e8a9"
format_stamp: "Formatted at 2025-10-27 02:49:43 on 5fd53c4cbb9d"
I20251027 02:49:43.350651 1733691 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-1/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-1/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:49:43.466457 1733691 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:49:43.468029 1733691 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:49:43.468571 1733691 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:49:43.557925 1733691 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.253:44983
I20251027 02:49:43.558065 1733783 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.253:44983 every 8 connection(s)
I20251027 02:49:43.560482 1733691 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-1/data/info.pb
I20251027 02:49:43.567430 1733784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:43.568946 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1733691
I20251027 02:49:43.569525 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-1/wal/instance
I20251027 02:49:43.573469 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.252:36319
--webserver_interface=127.152.231.252
--webserver_port=0
--builtin_ntp_servers=127.152.231.212:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.152.231.254:38201,127.152.231.253:44983,127.152.231.252:36319
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.cert
--webserver_private_key_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.key
--webserver_private_key_password_cmd=echo test with env {}
I20251027 02:49:43.579484 1733784 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:49:43.601038 1733784 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:49:43.615046 1733784 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } has no permanent_uuid. Determining permanent_uuid...
W20251027 02:49:43.617334 1733735 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.152.231.252:36319: connect: Connection refused (error 111)
W20251027 02:49:43.621824 1733784 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:36319: Network error: Client connection negotiation failed: client connection to 127.152.231.252:36319: connect: Connection refused (error 111)
I20251027 02:49:43.671341 1733784 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } attempt: 1
W20251027 02:49:43.674850 1733784 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:36319: Network error: Client connection negotiation failed: client connection to 127.152.231.252:36319: connect: Connection refused (error 111)
I20251027 02:49:43.743219 1733784 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } attempt: 2
W20251027 02:49:43.746703 1733784 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:36319: Network error: Client connection negotiation failed: client connection to 127.152.231.252:36319: connect: Connection refused (error 111)
I20251027 02:49:43.838047 1733784 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } attempt: 3
W20251027 02:49:43.841435 1733784 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:36319: Network error: Client connection negotiation failed: client connection to 127.152.231.252:36319: connect: Connection refused (error 111)
W20251027 02:49:43.921713 1733786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:49:43.922343 1733786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:49:43.922899 1733786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:49:43.955132 1733786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251027 02:49:43.955423 1733786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:49:43.955639 1733786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251027 02:49:43.955821 1733786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251027 02:49:43.984799 1733784 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } attempt: 4
W20251027 02:49:43.988835 1733784 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:36319: Network error: Client connection negotiation failed: client connection to 127.152.231.252:36319: connect: Connection refused (error 111)
I20251027 02:49:43.992730 1733786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.152.231.254:38201,127.152.231.253:44983,127.152.231.252:36319
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.252:36319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-2/data/info.pb
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.cert
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.252
--webserver_port=0
--webserver_private_key_file=<redacted>
--webserver_private_key_password_cmd=<redacted>
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:49:43.994107 1733786 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:49:43.995812 1733786 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251027 02:49:44.004347 1733689 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } attempt: 7
W20251027 02:49:44.008980 1733797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:49:44.016009 1733689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } has no permanent_uuid. Determining permanent_uuid...
W20251027 02:49:44.021725 1733689 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:36319: Network error: Client connection negotiation failed: client connection to 127.152.231.252:36319: connect: Connection refused (error 111)
I20251027 02:49:44.080135 1733689 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } attempt: 1
W20251027 02:49:44.084008 1733689 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:36319: Network error: Client connection negotiation failed: client connection to 127.152.231.252:36319: connect: Connection refused (error 111)
I20251027 02:49:44.165448 1733689 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } attempt: 2
W20251027 02:49:44.169670 1733689 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:36319: Network error: Client connection negotiation failed: client connection to 127.152.231.252:36319: connect: Connection refused (error 111)
I20251027 02:49:44.255102 1733689 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } attempt: 3
W20251027 02:49:44.258860 1733689 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:36319: Network error: Client connection negotiation failed: client connection to 127.152.231.252:36319: connect: Connection refused (error 111)
I20251027 02:49:44.288234 1733784 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } attempt: 5
W20251027 02:49:44.291736 1733784 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:36319: Network error: Client connection negotiation failed: client connection to 127.152.231.252:36319: connect: Connection refused (error 111)
I20251027 02:49:44.399267 1733689 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } attempt: 4
W20251027 02:49:44.403625 1733689 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:36319: Network error: Client connection negotiation failed: client connection to 127.152.231.252:36319: connect: Connection refused (error 111)
I20251027 02:49:44.687008 1733689 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } attempt: 5
W20251027 02:49:44.690762 1733689 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:36319: Network error: Client connection negotiation failed: client connection to 127.152.231.252:36319: connect: Connection refused (error 111)
I20251027 02:49:44.839139 1733784 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } attempt: 6
W20251027 02:49:44.845458 1733784 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:36319: Network error: Client connection negotiation failed: client connection to 127.152.231.252:36319: connect: Connection refused (error 111)
W20251027 02:49:44.009699 1733799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:49:45.178233 1733801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:49:45.180330 1733800 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1165 milliseconds
W20251027 02:49:45.180590 1733786 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.172s user 0.495s sys 0.675s
W20251027 02:49:45.181043 1733786 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.173s user 0.495s sys 0.675s
I20251027 02:49:45.181309 1733786 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:49:45.182490 1733786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:49:45.185947 1733786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:49:45.187338 1733786 hybrid_clock.cc:648] HybridClock initialized: now 1761533385187286 us; error 42 us; skew 500 ppm
I20251027 02:49:45.188302 1733786 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:49:45.217903 1733786 webserver.cc:492] Webserver started at https://127.152.231.252:44915/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:49:45.219274 1733786 fs_manager.cc:362] Metadata directory not provided
I20251027 02:49:45.219529 1733786 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:49:45.220111 1733786 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:49:45.226414 1733786 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-2/data/instance:
uuid: "0b4ee841b8324bc5bec17338b9abf918"
format_stamp: "Formatted at 2025-10-27 02:49:45 on 5fd53c4cbb9d"
I20251027 02:49:45.227576 1733786 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-2/wal/instance:
uuid: "0b4ee841b8324bc5bec17338b9abf918"
format_stamp: "Formatted at 2025-10-27 02:49:45 on 5fd53c4cbb9d"
I20251027 02:49:45.235781 1733786 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.001s sys 0.009s
I20251027 02:49:45.241676 1733814 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:49:45.242810 1733786 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20251027 02:49:45.243115 1733786 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-2/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-2/wal
uuid: "0b4ee841b8324bc5bec17338b9abf918"
format_stamp: "Formatted at 2025-10-27 02:49:45 on 5fd53c4cbb9d"
I20251027 02:49:45.243125 1733689 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } attempt: 6
I20251027 02:49:45.243417 1733786 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-2/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-2/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20251027 02:49:45.249987 1733689 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:36319: Network error: Client connection negotiation failed: client connection to 127.152.231.252:36319: connect: Connection refused (error 111)
I20251027 02:49:45.313639 1733786 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:49:45.315089 1733786 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:49:45.315582 1733786 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:49:45.403709 1733786 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.252:36319
I20251027 02:49:45.403837 1733867 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.252:36319 every 8 connection(s)
I20251027 02:49:45.406302 1733786 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-2/data/info.pb
I20251027 02:49:45.409600 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1733786
I20251027 02:49:45.410422 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/master-2/wal/instance
I20251027 02:49:45.412962 1733868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:45.425221 1733868 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:49:45.444849 1733868 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:49:45.453430 1733868 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:49:45.468544 1733868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918: Bootstrap starting.
I20251027 02:49:45.474236 1733868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:45.475742 1733868 log.cc:826] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918: Log is configured to *not* fsync() on all Append() calls
I20251027 02:49:45.480583 1733868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918: No bootstrap required, opened a new log
I20251027 02:49:45.496984 1733868 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } }
I20251027 02:49:45.497745 1733868 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:45.497964 1733868 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b4ee841b8324bc5bec17338b9abf918, State: Initialized, Role: FOLLOWER
I20251027 02:49:45.498667 1733868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } }
I20251027 02:49:45.501286 1733876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } } }
I20251027 02:49:45.502579 1733876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:49:45.503198 1733868 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [sys.catalog]: configured and running, proceeding with master startup.
W20251027 02:49:45.531610 1733887 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251027 02:49:45.532501 1733887 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251027 02:49:45.542160 1733876 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:49:45.542632 1733876 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } }
I20251027 02:49:45.544730 1733876 leader_election.cc:290] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 754fb6432e6f4fc89bfe76c08015f9c3 (127.152.231.254:38201), 5d6560dc68c84e38898fe92d1d47e8a9 (127.152.231.253:44983)
I20251027 02:49:45.527173 1733786 master_runner.cc:186] Error getting master registration for 127.152.231.254:38201: OK, instance_id { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" instance_seqno: 1761533381629300 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20251027 02:49:45.557091 1733786 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20251027 02:49:45.571310 1733822 tablet.cc:2378] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:49:45.905858 1733784 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } attempt: 7
I20251027 02:49:45.926466 1733784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9: Bootstrap starting.
I20251027 02:49:45.933310 1733784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:45.935299 1733784 log.cc:826] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9: Log is configured to *not* fsync() on all Append() calls
I20251027 02:49:45.940836 1733784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9: No bootstrap required, opened a new log
I20251027 02:49:45.968233 1733784 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } }
I20251027 02:49:45.969115 1733784 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:45.969417 1733784 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d6560dc68c84e38898fe92d1d47e8a9, State: Initialized, Role: FOLLOWER
I20251027 02:49:45.970290 1733784 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } }
I20251027 02:49:45.972783 1733891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } } }
I20251027 02:49:45.973878 1733891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:49:45.974761 1733784 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9 [sys.catalog]: configured and running, proceeding with master startup.
I20251027 02:49:45.974848 1733759 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b4ee841b8324bc5bec17338b9abf918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" is_pre_election: true
I20251027 02:49:45.975554 1733759 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b4ee841b8324bc5bec17338b9abf918 in term 0.
I20251027 02:49:45.977213 1733819 leader_election.cc:304] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b4ee841b8324bc5bec17338b9abf918, 5d6560dc68c84e38898fe92d1d47e8a9; no voters:
I20251027 02:49:45.978039 1733876 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20251027 02:49:45.978385 1733737 tablet.cc:2378] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:49:45.978559 1733876 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:49:45.978999 1733876 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:45.989347 1733876 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } }
I20251027 02:49:45.990856 1733876 leader_election.cc:290] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [CANDIDATE]: Term 1 election: Requested vote from peers 754fb6432e6f4fc89bfe76c08015f9c3 (127.152.231.254:38201), 5d6560dc68c84e38898fe92d1d47e8a9 (127.152.231.253:44983)
I20251027 02:49:45.991783 1733759 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b4ee841b8324bc5bec17338b9abf918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d6560dc68c84e38898fe92d1d47e8a9"
I20251027 02:49:45.992234 1733759 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:45.997812 1733759 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b4ee841b8324bc5bec17338b9abf918 in term 1.
I20251027 02:49:45.998732 1733819 leader_election.cc:304] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b4ee841b8324bc5bec17338b9abf918, 5d6560dc68c84e38898fe92d1d47e8a9; no voters:
I20251027 02:49:45.999341 1733876 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:49:46.001730 1733876 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [term 1 LEADER]: Becoming Leader. State: Replica: 0b4ee841b8324bc5bec17338b9abf918, State: Running, Role: LEADER
I20251027 02:49:46.003021 1733876 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } }
W20251027 02:49:46.009397 1733902 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251027 02:49:46.009690 1733902 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251027 02:49:46.013655 1733903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b4ee841b8324bc5bec17338b9abf918. Latest consensus state: current_term: 1 leader_uuid: "0b4ee841b8324bc5bec17338b9abf918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } } }
I20251027 02:49:46.014431 1733903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [sys.catalog]: This master's current role is: LEADER
I20251027 02:49:46.017935 1733906 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251027 02:49:46.024197 1733906 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251027 02:49:46.003490 1733691 master_runner.cc:186] Error getting master registration for 127.152.231.254:38201: OK, instance_id { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" instance_seqno: 1761533381629300 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20251027 02:49:46.027040 1733691 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20251027 02:49:46.038543 1733759 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9 [term 1 FOLLOWER]: Refusing update from remote peer 0b4ee841b8324bc5bec17338b9abf918: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:49:46.040148 1733903 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:49:46.059360 1733891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b4ee841b8324bc5bec17338b9abf918. Latest consensus state: current_term: 1 leader_uuid: "0b4ee841b8324bc5bec17338b9abf918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } } }
I20251027 02:49:46.060149 1733891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:49:46.061547 1733903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b4ee841b8324bc5bec17338b9abf918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } } }
I20251027 02:49:46.062204 1733903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [sys.catalog]: This master's current role is: LEADER
I20251027 02:49:46.064615 1733906 catalog_manager.cc:1357] Generated new cluster ID: de0db0d297854123a6bc7bf058097242
I20251027 02:49:46.064541 1733891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b4ee841b8324bc5bec17338b9abf918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } } }
I20251027 02:49:46.064841 1733906 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251027 02:49:46.065315 1733891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:49:46.083914 1733906 catalog_manager.cc:1380] Generated new certificate authority record
I20251027 02:49:46.085593 1733906 catalog_manager.cc:1514] Loading token signing keys...
I20251027 02:49:46.107434 1733906 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918: Generated new TSK 0
I20251027 02:49:46.108464 1733906 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251027 02:49:46.134102 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:49:46.136961 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.152.231.193:0
--local_ip_for_outbound_sockets=127.152.231.193
--webserver_interface=127.152.231.193
--webserver_port=0
--tserver_master_addrs=127.152.231.254:38201,127.152.231.253:44983,127.152.231.252:36319
--builtin_ntp_servers=127.152.231.212:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.cert
--webserver_private_key_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.key
--webserver_private_key_password_cmd=echo test with env {}
I20251027 02:49:46.283376 1733689 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } attempt: 7
I20251027 02:49:46.300613 1733689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3: Bootstrap starting.
I20251027 02:49:46.305246 1733689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:46.306643 1733689 log.cc:826] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3: Log is configured to *not* fsync() on all Append() calls
I20251027 02:49:46.311298 1733689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3: No bootstrap required, opened a new log
I20251027 02:49:46.328369 1733689 raft_consensus.cc:359] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } }
I20251027 02:49:46.328998 1733689 raft_consensus.cc:385] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:46.329270 1733689 raft_consensus.cc:740] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 754fb6432e6f4fc89bfe76c08015f9c3, State: Initialized, Role: FOLLOWER
I20251027 02:49:46.329988 1733689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } }
I20251027 02:49:46.332916 1733915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } } }
I20251027 02:49:46.334088 1733689 sys_catalog.cc:565] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3 [sys.catalog]: configured and running, proceeding with master startup.
I20251027 02:49:46.334033 1733915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:49:46.334193 1733664 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b4ee841b8324bc5bec17338b9abf918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" is_pre_election: true
I20251027 02:49:46.334298 1733663 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0b4ee841b8324bc5bec17338b9abf918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "754fb6432e6f4fc89bfe76c08015f9c3"
I20251027 02:49:46.334841 1733662 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:46.344789 1733662 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3 [term 1 FOLLOWER]: Refusing update from remote peer 0b4ee841b8324bc5bec17338b9abf918: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:49:46.346359 1733903 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:49:46.376082 1733915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b4ee841b8324bc5bec17338b9abf918. Latest consensus state: current_term: 1 leader_uuid: "0b4ee841b8324bc5bec17338b9abf918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } } }
I20251027 02:49:46.377061 1733915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:49:46.377331 1733929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0b4ee841b8324bc5bec17338b9abf918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } } }
I20251027 02:49:46.378961 1733929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:49:46.386299 1733933 catalog_manager.cc:1269] Loaded cluster ID: de0db0d297854123a6bc7bf058097242
I20251027 02:49:46.386515 1733933 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3: loading cluster ID for follower catalog manager: success
I20251027 02:49:46.391372 1733933 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3: acquiring CA information for follower catalog manager: success
I20251027 02:49:46.395378 1733933 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 754fb6432e6f4fc89bfe76c08015f9c3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20251027 02:49:46.527159 1733912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:49:46.527660 1733912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:49:46.528276 1733912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:49:46.560838 1733912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:49:46.561862 1733912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.152.231.193
I20251027 02:49:46.598357 1733912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.152.231.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-0/data/info.pb
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.cert
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.193
--webserver_port=0
--webserver_private_key_file=<redacted>
--webserver_private_key_password_cmd=<redacted>
--tserver_master_addrs=127.152.231.254:38201,127.152.231.253:44983,127.152.231.252:36319
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.152.231.193
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:49:46.599711 1733912 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:49:46.601531 1733912 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:49:46.615919 1733940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:49:46.929692 1733903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0b4ee841b8324bc5bec17338b9abf918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "754fb6432e6f4fc89bfe76c08015f9c3" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 38201 } } peers { permanent_uuid: "5d6560dc68c84e38898fe92d1d47e8a9" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 44983 } } peers { permanent_uuid: "0b4ee841b8324bc5bec17338b9abf918" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 36319 } } }
I20251027 02:49:46.930824 1733903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 [sys.catalog]: This master's current role is: LEADER
I20251027 02:49:47.014043 1733902 catalog_manager.cc:1269] Loaded cluster ID: de0db0d297854123a6bc7bf058097242
I20251027 02:49:47.014258 1733902 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9: loading cluster ID for follower catalog manager: success
I20251027 02:49:47.018603 1733902 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9: acquiring CA information for follower catalog manager: success
I20251027 02:49:47.021737 1733902 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 5d6560dc68c84e38898fe92d1d47e8a9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20251027 02:49:46.617311 1733941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:49:47.786947 1733942 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1166 milliseconds
W20251027 02:49:47.787384 1733912 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.172s user 0.525s sys 0.644s
W20251027 02:49:47.787803 1733912 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.172s user 0.526s sys 0.644s
W20251027 02:49:47.788443 1733943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:49:47.788494 1733912 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:49:47.789680 1733912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:49:47.792484 1733912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:49:47.793802 1733912 hybrid_clock.cc:648] HybridClock initialized: now 1761533387793747 us; error 43 us; skew 500 ppm
I20251027 02:49:47.794646 1733912 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:49:47.823326 1733912 webserver.cc:492] Webserver started at https://127.152.231.193:41067/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:49:47.824637 1733912 fs_manager.cc:362] Metadata directory not provided
I20251027 02:49:47.824887 1733912 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:49:47.825474 1733912 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:49:47.831602 1733912 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-0/data/instance:
uuid: "29af00c3fa84488f95905b28458dbbeb"
format_stamp: "Formatted at 2025-10-27 02:49:47 on 5fd53c4cbb9d"
I20251027 02:49:47.832683 1733912 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-0/wal/instance:
uuid: "29af00c3fa84488f95905b28458dbbeb"
format_stamp: "Formatted at 2025-10-27 02:49:47 on 5fd53c4cbb9d"
I20251027 02:49:47.840590 1733912 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20251027 02:49:47.846411 1733982 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:49:47.847461 1733912 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.000s sys 0.005s
I20251027 02:49:47.847745 1733912 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-0/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-0/wal
uuid: "29af00c3fa84488f95905b28458dbbeb"
format_stamp: "Formatted at 2025-10-27 02:49:47 on 5fd53c4cbb9d"
I20251027 02:49:47.848034 1733912 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:49:47.926261 1733912 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:49:47.927594 1733912 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:49:47.928036 1733912 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:49:47.931078 1733912 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251027 02:49:47.936317 1733912 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251027 02:49:47.936512 1733912 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251027 02:49:47.936703 1733912 ts_tablet_manager.cc:616] Registered 0 tablets
I20251027 02:49:47.936827 1733912 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251027 02:49:48.105767 1733912 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.193:43911
I20251027 02:49:48.105975 1734094 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.193:43911 every 8 connection(s)
I20251027 02:49:48.108083 1733912 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-0/data/info.pb
I20251027 02:49:48.117949 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1733912
I20251027 02:49:48.118366 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-0/wal/instance
I20251027 02:49:48.124423 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:49:48.125941 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.152.231.194:0
--local_ip_for_outbound_sockets=127.152.231.194
--webserver_interface=127.152.231.194
--webserver_port=0
--tserver_master_addrs=127.152.231.254:38201,127.152.231.253:44983,127.152.231.252:36319
--builtin_ntp_servers=127.152.231.212:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.cert
--webserver_private_key_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.key
--webserver_private_key_password_cmd=echo test with env {}
I20251027 02:49:48.135645 1734095 heartbeater.cc:344] Connected to a master server at 127.152.231.252:36319
I20251027 02:49:48.136066 1734095 heartbeater.cc:461] Registering TS with master...
I20251027 02:49:48.136710 1734096 heartbeater.cc:344] Connected to a master server at 127.152.231.253:44983
I20251027 02:49:48.136730 1734097 heartbeater.cc:344] Connected to a master server at 127.152.231.254:38201
I20251027 02:49:48.136997 1734096 heartbeater.cc:461] Registering TS with master...
I20251027 02:49:48.137135 1734097 heartbeater.cc:461] Registering TS with master...
I20251027 02:49:48.137329 1734095 heartbeater.cc:507] Master 127.152.231.252:36319 requested a full tablet report, sending...
I20251027 02:49:48.138113 1734097 heartbeater.cc:507] Master 127.152.231.254:38201 requested a full tablet report, sending...
I20251027 02:49:48.138144 1734096 heartbeater.cc:507] Master 127.152.231.253:44983 requested a full tablet report, sending...
I20251027 02:49:48.140039 1733833 ts_manager.cc:194] Registered new tserver with Master: 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911)
I20251027 02:49:48.140707 1733748 ts_manager.cc:194] Registered new tserver with Master: 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911)
I20251027 02:49:48.140707 1733654 ts_manager.cc:194] Registered new tserver with Master: 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911)
I20251027 02:49:48.141988 1733833 master_service.cc:502] Signed X509 certificate for tserver {username='jenkins-slave'} at 127.152.231.193:44157
W20251027 02:49:48.465732 1734105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:49:48.466223 1734105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:49:48.466811 1734105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:49:48.498773 1734105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:49:48.499730 1734105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.152.231.194
I20251027 02:49:48.535301 1734105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.152.231.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-1/data/info.pb
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.cert
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.194
--webserver_port=0
--webserver_private_key_file=<redacted>
--webserver_private_key_password_cmd=<redacted>
--tserver_master_addrs=127.152.231.254:38201,127.152.231.253:44983,127.152.231.252:36319
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.152.231.194
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:49:48.536640 1734105 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:49:48.538339 1734105 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:49:48.552300 1734112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:49:49.144889 1734095 heartbeater.cc:499] Master 127.152.231.252:36319 was elected leader, sending a full tablet report...
W20251027 02:49:48.553555 1734113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:49:49.681929 1734115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:49:49.683734 1734114 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1126 milliseconds
I20251027 02:49:49.683835 1734105 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:49:49.684981 1734105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:49:49.687880 1734105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:49:49.689196 1734105 hybrid_clock.cc:648] HybridClock initialized: now 1761533389689143 us; error 45 us; skew 500 ppm
I20251027 02:49:49.690040 1734105 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:49:49.717303 1734105 webserver.cc:492] Webserver started at https://127.152.231.194:36291/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:49:49.718592 1734105 fs_manager.cc:362] Metadata directory not provided
I20251027 02:49:49.718830 1734105 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:49:49.719354 1734105 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:49:49.725557 1734105 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-1/data/instance:
uuid: "c3f7def5fcba459bb5e0469d9ea5b690"
format_stamp: "Formatted at 2025-10-27 02:49:49 on 5fd53c4cbb9d"
I20251027 02:49:49.726858 1734105 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-1/wal/instance:
uuid: "c3f7def5fcba459bb5e0469d9ea5b690"
format_stamp: "Formatted at 2025-10-27 02:49:49 on 5fd53c4cbb9d"
I20251027 02:49:49.734758 1734105 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20251027 02:49:49.740631 1734124 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:49:49.741731 1734105 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20251027 02:49:49.742033 1734105 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-1/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-1/wal
uuid: "c3f7def5fcba459bb5e0469d9ea5b690"
format_stamp: "Formatted at 2025-10-27 02:49:49 on 5fd53c4cbb9d"
I20251027 02:49:49.742324 1734105 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:49:49.835628 1734105 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:49:49.836975 1734105 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:49:49.837540 1734105 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:49:49.840551 1734105 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251027 02:49:49.845702 1734105 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251027 02:49:49.845917 1734105 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251027 02:49:49.846117 1734105 ts_tablet_manager.cc:616] Registered 0 tablets
I20251027 02:49:49.846248 1734105 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251027 02:49:50.022622 1734105 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.194:36183
I20251027 02:49:50.022753 1734247 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.194:36183 every 8 connection(s)
I20251027 02:49:50.025306 1734105 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-1/data/info.pb
I20251027 02:49:50.033315 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1734105
I20251027 02:49:50.033715 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-1/wal/instance
I20251027 02:49:50.039824 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:49:50.041436 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.152.231.195:0
--local_ip_for_outbound_sockets=127.152.231.195
--webserver_interface=127.152.231.195
--webserver_port=0
--tserver_master_addrs=127.152.231.254:38201,127.152.231.253:44983,127.152.231.252:36319
--builtin_ntp_servers=127.152.231.212:40323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.cert
--webserver_private_key_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.key
--webserver_private_key_password_cmd=echo test with env {}
I20251027 02:49:50.053189 1734249 heartbeater.cc:344] Connected to a master server at 127.152.231.253:44983
I20251027 02:49:50.053593 1734249 heartbeater.cc:461] Registering TS with master...
I20251027 02:49:50.053828 1734248 heartbeater.cc:344] Connected to a master server at 127.152.231.252:36319
I20251027 02:49:50.054131 1734248 heartbeater.cc:461] Registering TS with master...
I20251027 02:49:50.054641 1734249 heartbeater.cc:507] Master 127.152.231.253:44983 requested a full tablet report, sending...
I20251027 02:49:50.055099 1734248 heartbeater.cc:507] Master 127.152.231.252:36319 requested a full tablet report, sending...
I20251027 02:49:50.057020 1733748 ts_manager.cc:194] Registered new tserver with Master: c3f7def5fcba459bb5e0469d9ea5b690 (127.152.231.194:36183)
I20251027 02:49:50.057193 1733833 ts_manager.cc:194] Registered new tserver with Master: c3f7def5fcba459bb5e0469d9ea5b690 (127.152.231.194:36183)
I20251027 02:49:50.057353 1734250 heartbeater.cc:344] Connected to a master server at 127.152.231.254:38201
I20251027 02:49:50.057623 1734250 heartbeater.cc:461] Registering TS with master...
I20251027 02:49:50.058382 1734250 heartbeater.cc:507] Master 127.152.231.254:38201 requested a full tablet report, sending...
I20251027 02:49:50.058789 1733833 master_service.cc:502] Signed X509 certificate for tserver {username='jenkins-slave'} at 127.152.231.194:36783
I20251027 02:49:50.060966 1733654 ts_manager.cc:194] Registered new tserver with Master: c3f7def5fcba459bb5e0469d9ea5b690 (127.152.231.194:36183)
W20251027 02:49:50.449318 1734258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:49:50.450001 1734258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:49:50.451056 1734258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:49:50.518678 1734258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:49:50.520546 1734258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.152.231.195
I20251027 02:49:50.592075 1734258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:40323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.152.231.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-2/data/info.pb
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/test.cert
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.195
--webserver_port=0
--webserver_private_key_file=<redacted>
--webserver_private_key_password_cmd=<redacted>
--tserver_master_addrs=127.152.231.254:38201,127.152.231.253:44983,127.152.231.252:36319
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.152.231.195
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:49:50.593981 1734258 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:49:50.596433 1734258 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:49:50.617009 1734376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:49:51.062110 1734248 heartbeater.cc:499] Master 127.152.231.252:36319 was elected leader, sending a full tablet report...
W20251027 02:49:50.617923 1734377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:49:52.017242 1734375 debug-util.cc:398] Leaking SignalData structure 0x7b0800034920 after lost signal to thread 1734258
W20251027 02:49:52.113401 1734258 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.496s user 0.640s sys 0.855s
W20251027 02:49:52.114054 1734258 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.497s user 0.640s sys 0.856s
W20251027 02:49:52.117467 1734379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:49:52.118927 1734378 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1496 milliseconds
I20251027 02:49:52.119033 1734258 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:49:52.120709 1734258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:49:52.124522 1734258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:49:52.125995 1734258 hybrid_clock.cc:648] HybridClock initialized: now 1761533392125934 us; error 53 us; skew 500 ppm
I20251027 02:49:52.127195 1734258 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:49:52.166098 1734258 webserver.cc:492] Webserver started at https://127.152.231.195:37693/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:49:52.167822 1734258 fs_manager.cc:362] Metadata directory not provided
I20251027 02:49:52.168201 1734258 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:49:52.168956 1734258 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:49:52.177381 1734258 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-2/data/instance:
uuid: "2a33e47daa87477b862ff20773137d55"
format_stamp: "Formatted at 2025-10-27 02:49:52 on 5fd53c4cbb9d"
I20251027 02:49:52.179205 1734258 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-2/wal/instance:
uuid: "2a33e47daa87477b862ff20773137d55"
format_stamp: "Formatted at 2025-10-27 02:49:52 on 5fd53c4cbb9d"
I20251027 02:49:52.190690 1734258 fs_manager.cc:696] Time spent creating directory manager: real 0.011s user 0.006s sys 0.006s
I20251027 02:49:52.199146 1734395 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:49:52.200569 1734258 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.001s sys 0.004s
I20251027 02:49:52.200990 1734258 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-2/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-2/wal
uuid: "2a33e47daa87477b862ff20773137d55"
format_stamp: "Formatted at 2025-10-27 02:49:52 on 5fd53c4cbb9d"
I20251027 02:49:52.201494 1734258 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:49:52.296841 1734258 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:49:52.298861 1734258 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:49:52.299561 1734258 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:49:52.303951 1734258 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251027 02:49:52.311313 1734258 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251027 02:49:52.311625 1734258 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251027 02:49:52.311957 1734258 ts_tablet_manager.cc:616] Registered 0 tablets
I20251027 02:49:52.312182 1734258 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251027 02:49:52.554278 1734258 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.195:33863
I20251027 02:49:52.554478 1734511 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.195:33863 every 8 connection(s)
I20251027 02:49:52.557953 1734258 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-2/data/info.pb
I20251027 02:49:52.567678 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1734258
I20251027 02:49:52.568251 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0/minicluster-data/ts-2/wal/instance
I20251027 02:49:52.602005 1734514 heartbeater.cc:344] Connected to a master server at 127.152.231.254:38201
I20251027 02:49:52.601994 1734512 heartbeater.cc:344] Connected to a master server at 127.152.231.252:36319
I20251027 02:49:52.602725 1734512 heartbeater.cc:461] Registering TS with master...
I20251027 02:49:52.602720 1734514 heartbeater.cc:461] Registering TS with master...
I20251027 02:49:52.604440 1734513 heartbeater.cc:344] Connected to a master server at 127.152.231.253:44983
I20251027 02:49:52.604691 1734514 heartbeater.cc:507] Master 127.152.231.254:38201 requested a full tablet report, sending...
I20251027 02:49:52.604674 1734512 heartbeater.cc:507] Master 127.152.231.252:36319 requested a full tablet report, sending...
I20251027 02:49:52.604810 1734513 heartbeater.cc:461] Registering TS with master...
I20251027 02:49:52.606106 1734513 heartbeater.cc:507] Master 127.152.231.253:44983 requested a full tablet report, sending...
I20251027 02:49:52.608095 1733833 ts_manager.cc:194] Registered new tserver with Master: 2a33e47daa87477b862ff20773137d55 (127.152.231.195:33863)
I20251027 02:49:52.608501 1733654 ts_manager.cc:194] Registered new tserver with Master: 2a33e47daa87477b862ff20773137d55 (127.152.231.195:33863)
I20251027 02:49:52.609910 1733833 master_service.cc:502] Signed X509 certificate for tserver {username='jenkins-slave'} at 127.152.231.195:53475
I20251027 02:49:52.610426 1733747 ts_manager.cc:194] Registered new tserver with Master: 2a33e47daa87477b862ff20773137d55 (127.152.231.195:33863)
I20251027 02:49:52.632217 1729439 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251027 02:49:52.647981 1729439 test_util.cc:276] Using random seed: -199166206
I20251027 02:49:52.683682 1733833 catalog_manager.cc:2257] Servicing CreateTable request from {username='jenkins-slave'} at 127.0.0.1:42798:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\004\001\000UUU\025\004\001\000\252\252\252*\004\001\000\377\377\377?\004\001\000TUUU\004\001\000\251\252\252j"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251027 02:49:52.687644 1733833 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251027 02:49:52.857856 1734173 tablet_service.cc:1505] Processing CreateTablet for tablet a7d21b4a0a0f4809822b7717e74a0710 (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION VALUES < 357913941
I20251027 02:49:52.860431 1734173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7d21b4a0a0f4809822b7717e74a0710. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.883376 1734172 tablet_service.cc:1505] Processing CreateTablet for tablet 2dfe0d3741334c98b3615811f8a993d7 (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20251027 02:49:52.886509 1734172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dfe0d3741334c98b3615811f8a993d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.886305 1734446 tablet_service.cc:1505] Processing CreateTablet for tablet 2dfe0d3741334c98b3615811f8a993d7 (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20251027 02:49:52.888790 1734446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dfe0d3741334c98b3615811f8a993d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.890689 1734171 tablet_service.cc:1505] Processing CreateTablet for tablet f3c4dd053dff44c28a4352a07dc4325e (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20251027 02:49:52.892279 1734171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3c4dd053dff44c28a4352a07dc4325e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.895362 1734030 tablet_service.cc:1505] Processing CreateTablet for tablet a7d21b4a0a0f4809822b7717e74a0710 (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION VALUES < 357913941
I20251027 02:49:52.884352 1734170 tablet_service.cc:1505] Processing CreateTablet for tablet 198ae238c5074f5390d31f3ca289c24d (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20251027 02:49:52.897800 1734028 tablet_service.cc:1505] Processing CreateTablet for tablet f3c4dd053dff44c28a4352a07dc4325e (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20251027 02:49:52.898766 1734170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 198ae238c5074f5390d31f3ca289c24d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.898562 1734442 tablet_service.cc:1505] Processing CreateTablet for tablet e347abcda9de4f348c443d4c39b2d7de (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20251027 02:49:52.900471 1734442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e347abcda9de4f348c443d4c39b2d7de. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.900274 1734025 tablet_service.cc:1505] Processing CreateTablet for tablet e347abcda9de4f348c443d4c39b2d7de (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20251027 02:49:52.902195 1734169 tablet_service.cc:1505] Processing CreateTablet for tablet a3f6339512a5402b96c0ded4bc9ed17a (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20251027 02:49:52.902604 1734444 tablet_service.cc:1505] Processing CreateTablet for tablet 198ae238c5074f5390d31f3ca289c24d (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20251027 02:49:52.903155 1734029 tablet_service.cc:1505] Processing CreateTablet for tablet 2dfe0d3741334c98b3615811f8a993d7 (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20251027 02:49:52.903651 1734169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3f6339512a5402b96c0ded4bc9ed17a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.903848 1734444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 198ae238c5074f5390d31f3ca289c24d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.904654 1734026 tablet_service.cc:1505] Processing CreateTablet for tablet a3f6339512a5402b96c0ded4bc9ed17a (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20251027 02:49:52.905997 1734027 tablet_service.cc:1505] Processing CreateTablet for tablet 198ae238c5074f5390d31f3ca289c24d (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20251027 02:49:52.909251 1734168 tablet_service.cc:1505] Processing CreateTablet for tablet e347abcda9de4f348c443d4c39b2d7de (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20251027 02:49:52.911188 1734168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e347abcda9de4f348c443d4c39b2d7de. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.911254 1734445 tablet_service.cc:1505] Processing CreateTablet for tablet f3c4dd053dff44c28a4352a07dc4325e (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20251027 02:49:52.911665 1734443 tablet_service.cc:1505] Processing CreateTablet for tablet a3f6339512a5402b96c0ded4bc9ed17a (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20251027 02:49:52.914049 1734445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3c4dd053dff44c28a4352a07dc4325e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.914122 1734030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7d21b4a0a0f4809822b7717e74a0710. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.915755 1734029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dfe0d3741334c98b3615811f8a993d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.917057 1734025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e347abcda9de4f348c443d4c39b2d7de. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.917728 1734026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3f6339512a5402b96c0ded4bc9ed17a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.916828 1734447 tablet_service.cc:1505] Processing CreateTablet for tablet a7d21b4a0a0f4809822b7717e74a0710 (DEFAULT_TABLE table=test-workload [id=91d480cdb2b2482e94c7bf509049db31]), partition=RANGE (key) PARTITION VALUES < 357913941
I20251027 02:49:52.918835 1734447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7d21b4a0a0f4809822b7717e74a0710. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.921406 1734027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 198ae238c5074f5390d31f3ca289c24d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.922335 1734443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3f6339512a5402b96c0ded4bc9ed17a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.923063 1734028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3c4dd053dff44c28a4352a07dc4325e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:49:52.986800 1734685 tablet_bootstrap.cc:492] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55: Bootstrap starting.
I20251027 02:49:52.994689 1734686 tablet_bootstrap.cc:492] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690: Bootstrap starting.
I20251027 02:49:52.996472 1734685 tablet_bootstrap.cc:654] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.009832 1734687 tablet_bootstrap.cc:492] T a7d21b4a0a0f4809822b7717e74a0710 P 29af00c3fa84488f95905b28458dbbeb: Bootstrap starting.
I20251027 02:49:53.011413 1734685 log.cc:826] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55: Log is configured to *not* fsync() on all Append() calls
I20251027 02:49:53.018796 1734686 tablet_bootstrap.cc:654] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.019196 1734687 tablet_bootstrap.cc:654] T a7d21b4a0a0f4809822b7717e74a0710 P 29af00c3fa84488f95905b28458dbbeb: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.021103 1734686 log.cc:826] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690: Log is configured to *not* fsync() on all Append() calls
I20251027 02:49:53.021296 1734687 log.cc:826] T a7d21b4a0a0f4809822b7717e74a0710 P 29af00c3fa84488f95905b28458dbbeb: Log is configured to *not* fsync() on all Append() calls
I20251027 02:49:53.033905 1734685 tablet_bootstrap.cc:492] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55: No bootstrap required, opened a new log
I20251027 02:49:53.034564 1734685 ts_tablet_manager.cc:1403] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55: Time spent bootstrapping tablet: real 0.048s user 0.015s sys 0.005s
I20251027 02:49:53.040813 1734686 tablet_bootstrap.cc:492] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690: No bootstrap required, opened a new log
I20251027 02:49:53.041463 1734686 ts_tablet_manager.cc:1403] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690: Time spent bootstrapping tablet: real 0.047s user 0.014s sys 0.005s
I20251027 02:49:53.052474 1734687 tablet_bootstrap.cc:492] T a7d21b4a0a0f4809822b7717e74a0710 P 29af00c3fa84488f95905b28458dbbeb: No bootstrap required, opened a new log
I20251027 02:49:53.053056 1734687 ts_tablet_manager.cc:1403] T a7d21b4a0a0f4809822b7717e74a0710 P 29af00c3fa84488f95905b28458dbbeb: Time spent bootstrapping tablet: real 0.044s user 0.015s sys 0.000s
I20251027 02:49:53.061383 1734685 raft_consensus.cc:359] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.062520 1734685 raft_consensus.cc:385] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.062881 1734685 raft_consensus.cc:740] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a33e47daa87477b862ff20773137d55, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.063905 1734685 consensus_queue.cc:260] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.068571 1734686 raft_consensus.cc:359] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.069618 1734686 raft_consensus.cc:385] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.069965 1734686 raft_consensus.cc:740] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3f7def5fcba459bb5e0469d9ea5b690, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.071313 1734686 consensus_queue.cc:260] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.083331 1734687 raft_consensus.cc:359] T a7d21b4a0a0f4809822b7717e74a0710 P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } } peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } }
I20251027 02:49:53.084344 1734687 raft_consensus.cc:385] T a7d21b4a0a0f4809822b7717e74a0710 P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.084651 1734687 raft_consensus.cc:740] T a7d21b4a0a0f4809822b7717e74a0710 P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29af00c3fa84488f95905b28458dbbeb, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.087088 1734686 ts_tablet_manager.cc:1434] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690: Time spent starting tablet: real 0.045s user 0.031s sys 0.011s
I20251027 02:49:53.088151 1734686 tablet_bootstrap.cc:492] T 2dfe0d3741334c98b3615811f8a993d7 P c3f7def5fcba459bb5e0469d9ea5b690: Bootstrap starting.
I20251027 02:49:53.094736 1734512 heartbeater.cc:499] Master 127.152.231.252:36319 was elected leader, sending a full tablet report...
I20251027 02:49:53.101845 1734687 consensus_queue.cc:260] T a7d21b4a0a0f4809822b7717e74a0710 P 29af00c3fa84488f95905b28458dbbeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } } peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } }
I20251027 02:49:53.104583 1734686 tablet_bootstrap.cc:654] T 2dfe0d3741334c98b3615811f8a993d7 P c3f7def5fcba459bb5e0469d9ea5b690: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.102681 1734685 ts_tablet_manager.cc:1434] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55: Time spent starting tablet: real 0.068s user 0.040s sys 0.020s
I20251027 02:49:53.106014 1734685 tablet_bootstrap.cc:492] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55: Bootstrap starting.
I20251027 02:49:53.111042 1734685 tablet_bootstrap.cc:654] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.122840 1734687 ts_tablet_manager.cc:1434] T a7d21b4a0a0f4809822b7717e74a0710 P 29af00c3fa84488f95905b28458dbbeb: Time spent starting tablet: real 0.069s user 0.031s sys 0.000s
I20251027 02:49:53.123916 1734687 tablet_bootstrap.cc:492] T f3c4dd053dff44c28a4352a07dc4325e P 29af00c3fa84488f95905b28458dbbeb: Bootstrap starting.
W20251027 02:49:53.126658 1734099 tablet.cc:2378] T a7d21b4a0a0f4809822b7717e74a0710 P 29af00c3fa84488f95905b28458dbbeb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:49:53.128531 1734686 tablet_bootstrap.cc:492] T 2dfe0d3741334c98b3615811f8a993d7 P c3f7def5fcba459bb5e0469d9ea5b690: No bootstrap required, opened a new log
I20251027 02:49:53.128957 1734686 ts_tablet_manager.cc:1403] T 2dfe0d3741334c98b3615811f8a993d7 P c3f7def5fcba459bb5e0469d9ea5b690: Time spent bootstrapping tablet: real 0.041s user 0.011s sys 0.005s
I20251027 02:49:53.129119 1734687 tablet_bootstrap.cc:654] T f3c4dd053dff44c28a4352a07dc4325e P 29af00c3fa84488f95905b28458dbbeb: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.131434 1734686 raft_consensus.cc:359] T 2dfe0d3741334c98b3615811f8a993d7 P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.132218 1734686 raft_consensus.cc:385] T 2dfe0d3741334c98b3615811f8a993d7 P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.132288 1734685 tablet_bootstrap.cc:492] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55: No bootstrap required, opened a new log
I20251027 02:49:53.132560 1734686 raft_consensus.cc:740] T 2dfe0d3741334c98b3615811f8a993d7 P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3f7def5fcba459bb5e0469d9ea5b690, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.132694 1734685 ts_tablet_manager.cc:1403] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55: Time spent bootstrapping tablet: real 0.027s user 0.009s sys 0.011s
I20251027 02:49:53.133283 1734686 consensus_queue.cc:260] T 2dfe0d3741334c98b3615811f8a993d7 P c3f7def5fcba459bb5e0469d9ea5b690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.135298 1734686 ts_tablet_manager.cc:1434] T 2dfe0d3741334c98b3615811f8a993d7 P c3f7def5fcba459bb5e0469d9ea5b690: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20251027 02:49:53.135792 1734685 raft_consensus.cc:359] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.136521 1734685 raft_consensus.cc:385] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.136837 1734685 raft_consensus.cc:740] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a33e47daa87477b862ff20773137d55, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.137557 1734685 consensus_queue.cc:260] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.139604 1734685 ts_tablet_manager.cc:1434] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20251027 02:49:53.141554 1734692 raft_consensus.cc:493] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:49:53.142189 1734692 raft_consensus.cc:515] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.145856 1734686 tablet_bootstrap.cc:492] T f3c4dd053dff44c28a4352a07dc4325e P c3f7def5fcba459bb5e0469d9ea5b690: Bootstrap starting.
I20251027 02:49:53.146672 1734685 tablet_bootstrap.cc:492] T e347abcda9de4f348c443d4c39b2d7de P 2a33e47daa87477b862ff20773137d55: Bootstrap starting.
I20251027 02:49:53.151858 1734685 tablet_bootstrap.cc:654] T e347abcda9de4f348c443d4c39b2d7de P 2a33e47daa87477b862ff20773137d55: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.151252 1734692 leader_election.cc:290] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911), 2a33e47daa87477b862ff20773137d55 (127.152.231.195:33863)
I20251027 02:49:53.155514 1734686 tablet_bootstrap.cc:654] T f3c4dd053dff44c28a4352a07dc4325e P c3f7def5fcba459bb5e0469d9ea5b690: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.159932 1734687 tablet_bootstrap.cc:492] T f3c4dd053dff44c28a4352a07dc4325e P 29af00c3fa84488f95905b28458dbbeb: No bootstrap required, opened a new log
I20251027 02:49:53.160384 1734687 ts_tablet_manager.cc:1403] T f3c4dd053dff44c28a4352a07dc4325e P 29af00c3fa84488f95905b28458dbbeb: Time spent bootstrapping tablet: real 0.037s user 0.013s sys 0.000s
I20251027 02:49:53.162851 1734687 raft_consensus.cc:359] T f3c4dd053dff44c28a4352a07dc4325e P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.163501 1734687 raft_consensus.cc:385] T f3c4dd053dff44c28a4352a07dc4325e P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.163758 1734687 raft_consensus.cc:740] T f3c4dd053dff44c28a4352a07dc4325e P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29af00c3fa84488f95905b28458dbbeb, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.164381 1734687 consensus_queue.cc:260] T f3c4dd053dff44c28a4352a07dc4325e P 29af00c3fa84488f95905b28458dbbeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.166364 1734687 ts_tablet_manager.cc:1434] T f3c4dd053dff44c28a4352a07dc4325e P 29af00c3fa84488f95905b28458dbbeb: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20251027 02:49:53.171195 1734685 tablet_bootstrap.cc:492] T e347abcda9de4f348c443d4c39b2d7de P 2a33e47daa87477b862ff20773137d55: No bootstrap required, opened a new log
I20251027 02:49:53.171620 1734685 ts_tablet_manager.cc:1403] T e347abcda9de4f348c443d4c39b2d7de P 2a33e47daa87477b862ff20773137d55: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.009s
I20251027 02:49:53.172231 1734687 tablet_bootstrap.cc:492] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb: Bootstrap starting.
I20251027 02:49:53.174620 1734685 raft_consensus.cc:359] T e347abcda9de4f348c443d4c39b2d7de P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.175252 1734685 raft_consensus.cc:385] T e347abcda9de4f348c443d4c39b2d7de P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.175511 1734685 raft_consensus.cc:740] T e347abcda9de4f348c443d4c39b2d7de P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a33e47daa87477b862ff20773137d55, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.176313 1734685 consensus_queue.cc:260] T e347abcda9de4f348c443d4c39b2d7de P 2a33e47daa87477b862ff20773137d55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.177269 1734686 tablet_bootstrap.cc:492] T f3c4dd053dff44c28a4352a07dc4325e P c3f7def5fcba459bb5e0469d9ea5b690: No bootstrap required, opened a new log
I20251027 02:49:53.177690 1734686 ts_tablet_manager.cc:1403] T f3c4dd053dff44c28a4352a07dc4325e P c3f7def5fcba459bb5e0469d9ea5b690: Time spent bootstrapping tablet: real 0.032s user 0.012s sys 0.004s
I20251027 02:49:53.178267 1734685 ts_tablet_manager.cc:1434] T e347abcda9de4f348c443d4c39b2d7de P 2a33e47daa87477b862ff20773137d55: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20251027 02:49:53.180317 1734686 raft_consensus.cc:359] T f3c4dd053dff44c28a4352a07dc4325e P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.180913 1734686 raft_consensus.cc:385] T f3c4dd053dff44c28a4352a07dc4325e P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.181198 1734686 raft_consensus.cc:740] T f3c4dd053dff44c28a4352a07dc4325e P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3f7def5fcba459bb5e0469d9ea5b690, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.181823 1734686 consensus_queue.cc:260] T f3c4dd053dff44c28a4352a07dc4325e P c3f7def5fcba459bb5e0469d9ea5b690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.183912 1734686 ts_tablet_manager.cc:1434] T f3c4dd053dff44c28a4352a07dc4325e P c3f7def5fcba459bb5e0469d9ea5b690: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20251027 02:49:53.189507 1734467 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e347abcda9de4f348c443d4c39b2d7de" candidate_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a33e47daa87477b862ff20773137d55" is_pre_election: true
I20251027 02:49:53.189837 1734685 tablet_bootstrap.cc:492] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55: Bootstrap starting.
I20251027 02:49:53.190294 1734687 tablet_bootstrap.cc:654] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.190541 1734467 raft_consensus.cc:2468] T e347abcda9de4f348c443d4c39b2d7de P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3f7def5fcba459bb5e0469d9ea5b690 in term 0.
I20251027 02:49:53.190946 1734686 tablet_bootstrap.cc:492] T 198ae238c5074f5390d31f3ca289c24d P c3f7def5fcba459bb5e0469d9ea5b690: Bootstrap starting.
I20251027 02:49:53.195258 1734685 tablet_bootstrap.cc:654] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.195222 1734127 leader_election.cc:304] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a33e47daa87477b862ff20773137d55, c3f7def5fcba459bb5e0469d9ea5b690; no voters:
I20251027 02:49:53.197468 1734692 raft_consensus.cc:2804] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:49:53.197786 1734686 tablet_bootstrap.cc:654] T 198ae238c5074f5390d31f3ca289c24d P c3f7def5fcba459bb5e0469d9ea5b690: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.197880 1734692 raft_consensus.cc:493] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:49:53.198735 1734692 raft_consensus.cc:3060] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:53.200522 1734050 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e347abcda9de4f348c443d4c39b2d7de" candidate_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29af00c3fa84488f95905b28458dbbeb" is_pre_election: true
W20251027 02:49:53.202697 1734126 leader_election.cc:343] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911): Illegal state: must be running to vote when last-logged opid is not known
I20251027 02:49:53.211477 1734687 tablet_bootstrap.cc:492] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb: No bootstrap required, opened a new log
I20251027 02:49:53.211925 1734687 ts_tablet_manager.cc:1403] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb: Time spent bootstrapping tablet: real 0.040s user 0.013s sys 0.004s
I20251027 02:49:53.214363 1734687 raft_consensus.cc:359] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.215011 1734687 raft_consensus.cc:385] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.215297 1734687 raft_consensus.cc:740] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29af00c3fa84488f95905b28458dbbeb, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.215708 1734685 tablet_bootstrap.cc:492] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55: No bootstrap required, opened a new log
I20251027 02:49:53.216130 1734685 ts_tablet_manager.cc:1403] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.009s
I20251027 02:49:53.215916 1734687 consensus_queue.cc:260] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.218034 1734687 ts_tablet_manager.cc:1434] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20251027 02:49:53.218484 1734685 raft_consensus.cc:359] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.218621 1734692 raft_consensus.cc:515] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.219363 1734685 raft_consensus.cc:385] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.219683 1734685 raft_consensus.cc:740] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a33e47daa87477b862ff20773137d55, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.220381 1734692 leader_election.cc:290] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [CANDIDATE]: Term 1 election: Requested vote from peers 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911), 2a33e47daa87477b862ff20773137d55 (127.152.231.195:33863)
I20251027 02:49:53.220449 1734685 consensus_queue.cc:260] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.221302 1734687 tablet_bootstrap.cc:492] T 198ae238c5074f5390d31f3ca289c24d P 29af00c3fa84488f95905b28458dbbeb: Bootstrap starting.
I20251027 02:49:53.221331 1734050 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e347abcda9de4f348c443d4c39b2d7de" candidate_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29af00c3fa84488f95905b28458dbbeb"
I20251027 02:49:53.221872 1734467 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e347abcda9de4f348c443d4c39b2d7de" candidate_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a33e47daa87477b862ff20773137d55"
I20251027 02:49:53.222446 1734467 raft_consensus.cc:3060] T e347abcda9de4f348c443d4c39b2d7de P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Advancing to term 1
W20251027 02:49:53.222654 1734126 leader_election.cc:343] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911): Illegal state: must be running to vote when last-logged opid is not known
I20251027 02:49:53.224696 1734685 ts_tablet_manager.cc:1434] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20251027 02:49:53.225709 1734685 tablet_bootstrap.cc:492] T 2dfe0d3741334c98b3615811f8a993d7 P 2a33e47daa87477b862ff20773137d55: Bootstrap starting.
I20251027 02:49:53.226794 1734687 tablet_bootstrap.cc:654] T 198ae238c5074f5390d31f3ca289c24d P 29af00c3fa84488f95905b28458dbbeb: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.228531 1734691 raft_consensus.cc:493] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:49:53.230288 1734467 raft_consensus.cc:2468] T e347abcda9de4f348c443d4c39b2d7de P 2a33e47daa87477b862ff20773137d55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3f7def5fcba459bb5e0469d9ea5b690 in term 1.
I20251027 02:49:53.229444 1734691 raft_consensus.cc:515] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.231710 1734127 leader_election.cc:304] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a33e47daa87477b862ff20773137d55, c3f7def5fcba459bb5e0469d9ea5b690; no voters: 29af00c3fa84488f95905b28458dbbeb
I20251027 02:49:53.232646 1734692 raft_consensus.cc:2804] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:49:53.233778 1734691 leader_election.cc:290] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911), c3f7def5fcba459bb5e0469d9ea5b690 (127.152.231.194:36183)
I20251027 02:49:53.235754 1734686 tablet_bootstrap.cc:492] T 198ae238c5074f5390d31f3ca289c24d P c3f7def5fcba459bb5e0469d9ea5b690: No bootstrap required, opened a new log
I20251027 02:49:53.236195 1734686 ts_tablet_manager.cc:1403] T 198ae238c5074f5390d31f3ca289c24d P c3f7def5fcba459bb5e0469d9ea5b690: Time spent bootstrapping tablet: real 0.045s user 0.013s sys 0.004s
I20251027 02:49:53.238481 1734686 raft_consensus.cc:359] T 198ae238c5074f5390d31f3ca289c24d P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.239105 1734686 raft_consensus.cc:385] T 198ae238c5074f5390d31f3ca289c24d P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.239356 1734686 raft_consensus.cc:740] T 198ae238c5074f5390d31f3ca289c24d P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3f7def5fcba459bb5e0469d9ea5b690, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.239938 1734686 consensus_queue.cc:260] T 198ae238c5074f5390d31f3ca289c24d P c3f7def5fcba459bb5e0469d9ea5b690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.240634 1734685 tablet_bootstrap.cc:654] T 2dfe0d3741334c98b3615811f8a993d7 P 2a33e47daa87477b862ff20773137d55: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.249214 1734692 raft_consensus.cc:697] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [term 1 LEADER]: Becoming Leader. State: Replica: c3f7def5fcba459bb5e0469d9ea5b690, State: Running, Role: LEADER
I20251027 02:49:53.250245 1734692 consensus_queue.cc:237] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.251812 1734687 tablet_bootstrap.cc:492] T 198ae238c5074f5390d31f3ca289c24d P 29af00c3fa84488f95905b28458dbbeb: No bootstrap required, opened a new log
I20251027 02:49:53.252204 1734687 ts_tablet_manager.cc:1403] T 198ae238c5074f5390d31f3ca289c24d P 29af00c3fa84488f95905b28458dbbeb: Time spent bootstrapping tablet: real 0.031s user 0.010s sys 0.000s
I20251027 02:49:53.254297 1734687 raft_consensus.cc:359] T 198ae238c5074f5390d31f3ca289c24d P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.254894 1734687 raft_consensus.cc:385] T 198ae238c5074f5390d31f3ca289c24d P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.255127 1734687 raft_consensus.cc:740] T 198ae238c5074f5390d31f3ca289c24d P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29af00c3fa84488f95905b28458dbbeb, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.255620 1734687 consensus_queue.cc:260] T 198ae238c5074f5390d31f3ca289c24d P 29af00c3fa84488f95905b28458dbbeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.257436 1734687 ts_tablet_manager.cc:1434] T 198ae238c5074f5390d31f3ca289c24d P 29af00c3fa84488f95905b28458dbbeb: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20251027 02:49:53.258319 1734686 ts_tablet_manager.cc:1434] T 198ae238c5074f5390d31f3ca289c24d P c3f7def5fcba459bb5e0469d9ea5b690: Time spent starting tablet: real 0.022s user 0.010s sys 0.000s
I20251027 02:49:53.258472 1734687 tablet_bootstrap.cc:492] T a3f6339512a5402b96c0ded4bc9ed17a P 29af00c3fa84488f95905b28458dbbeb: Bootstrap starting.
I20251027 02:49:53.259197 1734686 tablet_bootstrap.cc:492] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690: Bootstrap starting.
I20251027 02:49:53.263995 1734687 tablet_bootstrap.cc:654] T a3f6339512a5402b96c0ded4bc9ed17a P 29af00c3fa84488f95905b28458dbbeb: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.265417 1734686 tablet_bootstrap.cc:654] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.272253 1734685 tablet_bootstrap.cc:492] T 2dfe0d3741334c98b3615811f8a993d7 P 2a33e47daa87477b862ff20773137d55: No bootstrap required, opened a new log
I20251027 02:49:53.272764 1734685 ts_tablet_manager.cc:1403] T 2dfe0d3741334c98b3615811f8a993d7 P 2a33e47daa87477b862ff20773137d55: Time spent bootstrapping tablet: real 0.047s user 0.021s sys 0.014s
I20251027 02:49:53.274673 1734050 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "198ae238c5074f5390d31f3ca289c24d" candidate_uuid: "2a33e47daa87477b862ff20773137d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29af00c3fa84488f95905b28458dbbeb" is_pre_election: true
I20251027 02:49:53.275269 1734685 raft_consensus.cc:359] T 2dfe0d3741334c98b3615811f8a993d7 P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.275444 1734050 raft_consensus.cc:2468] T 198ae238c5074f5390d31f3ca289c24d P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a33e47daa87477b862ff20773137d55 in term 0.
I20251027 02:49:53.275872 1734685 raft_consensus.cc:385] T 2dfe0d3741334c98b3615811f8a993d7 P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.276129 1734685 raft_consensus.cc:740] T 2dfe0d3741334c98b3615811f8a993d7 P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a33e47daa87477b862ff20773137d55, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.276662 1734401 leader_election.cc:304] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29af00c3fa84488f95905b28458dbbeb, 2a33e47daa87477b862ff20773137d55; no voters:
I20251027 02:49:53.276772 1734685 consensus_queue.cc:260] T 2dfe0d3741334c98b3615811f8a993d7 P 2a33e47daa87477b862ff20773137d55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.277849 1734691 raft_consensus.cc:2804] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:49:53.278180 1734691 raft_consensus.cc:493] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:49:53.278496 1734691 raft_consensus.cc:3060] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:53.286712 1734691 raft_consensus.cc:515] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.288504 1734691 leader_election.cc:290] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [CANDIDATE]: Term 1 election: Requested vote from peers 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911), c3f7def5fcba459bb5e0469d9ea5b690 (127.152.231.194:36183)
I20251027 02:49:53.289713 1734050 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "198ae238c5074f5390d31f3ca289c24d" candidate_uuid: "2a33e47daa87477b862ff20773137d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29af00c3fa84488f95905b28458dbbeb"
I20251027 02:49:53.290352 1734050 raft_consensus.cc:3060] T 198ae238c5074f5390d31f3ca289c24d P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Advancing to term 1
W20251027 02:49:53.291003 1734252 tablet.cc:2378] T 198ae238c5074f5390d31f3ca289c24d P c3f7def5fcba459bb5e0469d9ea5b690: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:49:53.295730 1733833 catalog_manager.cc:5649] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3f7def5fcba459bb5e0469d9ea5b690 (127.152.231.194). New cstate: current_term: 1 leader_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } health_report { overall_health: UNKNOWN } } }
I20251027 02:49:53.298048 1734050 raft_consensus.cc:2468] T 198ae238c5074f5390d31f3ca289c24d P 29af00c3fa84488f95905b28458dbbeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a33e47daa87477b862ff20773137d55 in term 1.
I20251027 02:49:53.297586 1734193 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "198ae238c5074f5390d31f3ca289c24d" candidate_uuid: "2a33e47daa87477b862ff20773137d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" is_pre_election: true
I20251027 02:49:53.299008 1734401 leader_election.cc:304] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29af00c3fa84488f95905b28458dbbeb, 2a33e47daa87477b862ff20773137d55; no voters:
I20251027 02:49:53.300040 1734685 ts_tablet_manager.cc:1434] T 2dfe0d3741334c98b3615811f8a993d7 P 2a33e47daa87477b862ff20773137d55: Time spent starting tablet: real 0.027s user 0.008s sys 0.000s
I20251027 02:49:53.299376 1734193 raft_consensus.cc:2468] T 198ae238c5074f5390d31f3ca289c24d P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a33e47daa87477b862ff20773137d55 in term 0.
I20251027 02:49:53.300241 1734686 tablet_bootstrap.cc:492] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690: No bootstrap required, opened a new log
I20251027 02:49:53.300740 1734686 ts_tablet_manager.cc:1403] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690: Time spent bootstrapping tablet: real 0.042s user 0.011s sys 0.005s
I20251027 02:49:53.297037 1734192 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "198ae238c5074f5390d31f3ca289c24d" candidate_uuid: "2a33e47daa87477b862ff20773137d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3f7def5fcba459bb5e0469d9ea5b690"
I20251027 02:49:53.301095 1734685 tablet_bootstrap.cc:492] T a7d21b4a0a0f4809822b7717e74a0710 P 2a33e47daa87477b862ff20773137d55: Bootstrap starting.
I20251027 02:49:53.300459 1734713 raft_consensus.cc:2804] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:49:53.301273 1734192 raft_consensus.cc:3060] T 198ae238c5074f5390d31f3ca289c24d P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:53.301697 1734713 raft_consensus.cc:697] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [term 1 LEADER]: Becoming Leader. State: Replica: 2a33e47daa87477b862ff20773137d55, State: Running, Role: LEADER
I20251027 02:49:53.302546 1734687 tablet_bootstrap.cc:492] T a3f6339512a5402b96c0ded4bc9ed17a P 29af00c3fa84488f95905b28458dbbeb: No bootstrap required, opened a new log
I20251027 02:49:53.302937 1734687 ts_tablet_manager.cc:1403] T a3f6339512a5402b96c0ded4bc9ed17a P 29af00c3fa84488f95905b28458dbbeb: Time spent bootstrapping tablet: real 0.045s user 0.014s sys 0.012s
I20251027 02:49:53.302729 1734713 consensus_queue.cc:237] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.303932 1734686 raft_consensus.cc:359] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } } peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } }
I20251027 02:49:53.304735 1734686 raft_consensus.cc:385] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.305050 1734686 raft_consensus.cc:740] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3f7def5fcba459bb5e0469d9ea5b690, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.305349 1734687 raft_consensus.cc:359] T a3f6339512a5402b96c0ded4bc9ed17a P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.305987 1734687 raft_consensus.cc:385] T a3f6339512a5402b96c0ded4bc9ed17a P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.305757 1734686 consensus_queue.cc:260] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } } peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } }
I20251027 02:49:53.306298 1734687 raft_consensus.cc:740] T a3f6339512a5402b96c0ded4bc9ed17a P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29af00c3fa84488f95905b28458dbbeb, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.306897 1734687 consensus_queue.cc:260] T a3f6339512a5402b96c0ded4bc9ed17a P 29af00c3fa84488f95905b28458dbbeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.308034 1734686 ts_tablet_manager.cc:1434] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20251027 02:49:53.308384 1734685 tablet_bootstrap.cc:654] T a7d21b4a0a0f4809822b7717e74a0710 P 2a33e47daa87477b862ff20773137d55: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.308804 1734687 ts_tablet_manager.cc:1434] T a3f6339512a5402b96c0ded4bc9ed17a P 29af00c3fa84488f95905b28458dbbeb: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20251027 02:49:53.308725 1734192 raft_consensus.cc:2468] T 198ae238c5074f5390d31f3ca289c24d P c3f7def5fcba459bb5e0469d9ea5b690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a33e47daa87477b862ff20773137d55 in term 1.
I20251027 02:49:53.308943 1734686 tablet_bootstrap.cc:492] T a3f6339512a5402b96c0ded4bc9ed17a P c3f7def5fcba459bb5e0469d9ea5b690: Bootstrap starting.
I20251027 02:49:53.309789 1734687 tablet_bootstrap.cc:492] T e347abcda9de4f348c443d4c39b2d7de P 29af00c3fa84488f95905b28458dbbeb: Bootstrap starting.
I20251027 02:49:53.310999 1734691 raft_consensus.cc:493] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:49:53.311403 1734691 raft_consensus.cc:515] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.312968 1734691 leader_election.cc:290] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911), c3f7def5fcba459bb5e0469d9ea5b690 (127.152.231.194:36183)
I20251027 02:49:53.313380 1734693 raft_consensus.cc:493] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:49:53.317533 1734692 raft_consensus.cc:493] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:49:53.318015 1734692 raft_consensus.cc:515] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } } peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } }
I20251027 02:49:53.319782 1734692 leader_election.cc:290] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a33e47daa87477b862ff20773137d55 (127.152.231.195:33863), 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911)
I20251027 02:49:53.320542 1734467 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a7d21b4a0a0f4809822b7717e74a0710" candidate_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a33e47daa87477b862ff20773137d55" is_pre_election: true
W20251027 02:49:53.322891 1734127 leader_election.cc:343] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a33e47daa87477b862ff20773137d55 (127.152.231.195:33863): Illegal state: must be running to vote when last-logged opid is not known
I20251027 02:49:53.323247 1734192 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3f6339512a5402b96c0ded4bc9ed17a" candidate_uuid: "2a33e47daa87477b862ff20773137d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" is_pre_election: true
W20251027 02:49:53.324697 1734401 leader_election.cc:343] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3f7def5fcba459bb5e0469d9ea5b690 (127.152.231.194:36183): Illegal state: must be running to vote when last-logged opid is not known
I20251027 02:49:53.313866 1734693 raft_consensus.cc:515] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.327325 1734050 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3f6339512a5402b96c0ded4bc9ed17a" candidate_uuid: "2a33e47daa87477b862ff20773137d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29af00c3fa84488f95905b28458dbbeb" is_pre_election: true
I20251027 02:49:53.327643 1734687 tablet_bootstrap.cc:654] T e347abcda9de4f348c443d4c39b2d7de P 29af00c3fa84488f95905b28458dbbeb: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.327905 1734050 raft_consensus.cc:2468] T a3f6339512a5402b96c0ded4bc9ed17a P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a33e47daa87477b862ff20773137d55 in term 0.
I20251027 02:49:53.328858 1734401 leader_election.cc:304] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29af00c3fa84488f95905b28458dbbeb, 2a33e47daa87477b862ff20773137d55; no voters: c3f7def5fcba459bb5e0469d9ea5b690
W20251027 02:49:53.329713 1734516 tablet.cc:2378] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:49:53.329715 1734691 raft_consensus.cc:2804] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:49:53.330247 1734691 raft_consensus.cc:493] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:49:53.330547 1734050 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a7d21b4a0a0f4809822b7717e74a0710" candidate_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29af00c3fa84488f95905b28458dbbeb" is_pre_election: true
I20251027 02:49:53.330624 1734691 raft_consensus.cc:3060] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:53.331107 1734686 tablet_bootstrap.cc:654] T a3f6339512a5402b96c0ded4bc9ed17a P c3f7def5fcba459bb5e0469d9ea5b690: Neither blocks nor log segments found. Creating new log.
I20251027 02:49:53.331171 1734050 raft_consensus.cc:2468] T a7d21b4a0a0f4809822b7717e74a0710 P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3f7def5fcba459bb5e0469d9ea5b690 in term 0.
I20251027 02:49:53.332101 1734126 leader_election.cc:304] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29af00c3fa84488f95905b28458dbbeb, c3f7def5fcba459bb5e0469d9ea5b690; no voters: 2a33e47daa87477b862ff20773137d55
I20251027 02:49:53.332938 1734692 raft_consensus.cc:2804] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:49:53.333297 1734692 raft_consensus.cc:493] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:49:53.333585 1734692 raft_consensus.cc:3060] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:53.333688 1734693 leader_election.cc:290] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3f7def5fcba459bb5e0469d9ea5b690 (127.152.231.194:36183), 2a33e47daa87477b862ff20773137d55 (127.152.231.195:33863)
I20251027 02:49:53.337196 1734691 raft_consensus.cc:515] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.339082 1734691 leader_election.cc:290] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [CANDIDATE]: Term 1 election: Requested vote from peers 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911), c3f7def5fcba459bb5e0469d9ea5b690 (127.152.231.194:36183)
I20251027 02:49:53.339520 1734692 raft_consensus.cc:515] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } } peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } }
I20251027 02:49:53.340803 1734685 tablet_bootstrap.cc:492] T a7d21b4a0a0f4809822b7717e74a0710 P 2a33e47daa87477b862ff20773137d55: No bootstrap required, opened a new log
I20251027 02:49:53.340934 1734192 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3f6339512a5402b96c0ded4bc9ed17a" candidate_uuid: "2a33e47daa87477b862ff20773137d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3f7def5fcba459bb5e0469d9ea5b690"
I20251027 02:49:53.341274 1734685 ts_tablet_manager.cc:1403] T a7d21b4a0a0f4809822b7717e74a0710 P 2a33e47daa87477b862ff20773137d55: Time spent bootstrapping tablet: real 0.040s user 0.014s sys 0.006s
I20251027 02:49:53.341930 1734692 leader_election.cc:290] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [CANDIDATE]: Term 1 election: Requested vote from peers 2a33e47daa87477b862ff20773137d55 (127.152.231.195:33863), 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911)
W20251027 02:49:53.342217 1734401 leader_election.cc:343] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c3f7def5fcba459bb5e0469d9ea5b690 (127.152.231.194:36183): Illegal state: must be running to vote when last-logged opid is not known
I20251027 02:49:53.342720 1734467 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a7d21b4a0a0f4809822b7717e74a0710" candidate_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a33e47daa87477b862ff20773137d55"
I20251027 02:49:53.343492 1734050 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a7d21b4a0a0f4809822b7717e74a0710" candidate_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29af00c3fa84488f95905b28458dbbeb"
I20251027 02:49:53.343994 1734050 raft_consensus.cc:3060] T a7d21b4a0a0f4809822b7717e74a0710 P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Advancing to term 1
W20251027 02:49:53.343899 1734127 leader_election.cc:343] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 2a33e47daa87477b862ff20773137d55 (127.152.231.195:33863): Illegal state: must be running to vote when last-logged opid is not known
I20251027 02:49:53.343703 1734685 raft_consensus.cc:359] T a7d21b4a0a0f4809822b7717e74a0710 P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } } peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } }
I20251027 02:49:53.344312 1734685 raft_consensus.cc:385] T a7d21b4a0a0f4809822b7717e74a0710 P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.344585 1734685 raft_consensus.cc:740] T a7d21b4a0a0f4809822b7717e74a0710 P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a33e47daa87477b862ff20773137d55, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.345266 1734685 consensus_queue.cc:260] T a7d21b4a0a0f4809822b7717e74a0710 P 2a33e47daa87477b862ff20773137d55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } } peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } }
I20251027 02:49:53.347375 1734685 ts_tablet_manager.cc:1434] T a7d21b4a0a0f4809822b7717e74a0710 P 2a33e47daa87477b862ff20773137d55: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20251027 02:49:53.348387 1734686 tablet_bootstrap.cc:492] T a3f6339512a5402b96c0ded4bc9ed17a P c3f7def5fcba459bb5e0469d9ea5b690: No bootstrap required, opened a new log
I20251027 02:49:53.348862 1734686 ts_tablet_manager.cc:1403] T a3f6339512a5402b96c0ded4bc9ed17a P c3f7def5fcba459bb5e0469d9ea5b690: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.017s
I20251027 02:49:53.351351 1734686 raft_consensus.cc:359] T a3f6339512a5402b96c0ded4bc9ed17a P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.352037 1734686 raft_consensus.cc:385] T a3f6339512a5402b96c0ded4bc9ed17a P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.352365 1734686 raft_consensus.cc:740] T a3f6339512a5402b96c0ded4bc9ed17a P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3f7def5fcba459bb5e0469d9ea5b690, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.353045 1734691 raft_consensus.cc:493] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:49:53.353057 1734686 consensus_queue.cc:260] T a3f6339512a5402b96c0ded4bc9ed17a P c3f7def5fcba459bb5e0469d9ea5b690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.353492 1734691 raft_consensus.cc:515] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.355217 1734686 ts_tablet_manager.cc:1434] T a3f6339512a5402b96c0ded4bc9ed17a P c3f7def5fcba459bb5e0469d9ea5b690: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20251027 02:49:53.355728 1734691 leader_election.cc:290] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911), c3f7def5fcba459bb5e0469d9ea5b690 (127.152.231.194:36183)
I20251027 02:49:53.361478 1734050 raft_consensus.cc:2468] T a7d21b4a0a0f4809822b7717e74a0710 P 29af00c3fa84488f95905b28458dbbeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3f7def5fcba459bb5e0469d9ea5b690 in term 1.
I20251027 02:49:53.362615 1734126 leader_election.cc:304] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29af00c3fa84488f95905b28458dbbeb, c3f7def5fcba459bb5e0469d9ea5b690; no voters: 2a33e47daa87477b862ff20773137d55
I20251027 02:49:53.361403 1734049 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3f6339512a5402b96c0ded4bc9ed17a" candidate_uuid: "2a33e47daa87477b862ff20773137d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29af00c3fa84488f95905b28458dbbeb"
I20251027 02:49:53.363310 1734692 raft_consensus.cc:2804] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:49:53.363770 1734692 raft_consensus.cc:697] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [term 1 LEADER]: Becoming Leader. State: Replica: c3f7def5fcba459bb5e0469d9ea5b690, State: Running, Role: LEADER
I20251027 02:49:53.363334 1734049 raft_consensus.cc:3060] T a3f6339512a5402b96c0ded4bc9ed17a P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:53.364526 1734692 consensus_queue.cc:237] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } } peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } }
I20251027 02:49:53.369382 1734192 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f3c4dd053dff44c28a4352a07dc4325e" candidate_uuid: "2a33e47daa87477b862ff20773137d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" is_pre_election: true
I20251027 02:49:53.370033 1734192 raft_consensus.cc:2468] T f3c4dd053dff44c28a4352a07dc4325e P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a33e47daa87477b862ff20773137d55 in term 0.
I20251027 02:49:53.371227 1734401 leader_election.cc:304] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a33e47daa87477b862ff20773137d55, c3f7def5fcba459bb5e0469d9ea5b690; no voters:
I20251027 02:49:53.372092 1734691 raft_consensus.cc:2804] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:49:53.372426 1734691 raft_consensus.cc:493] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:49:53.372725 1734691 raft_consensus.cc:3060] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:53.373198 1734687 tablet_bootstrap.cc:492] T e347abcda9de4f348c443d4c39b2d7de P 29af00c3fa84488f95905b28458dbbeb: No bootstrap required, opened a new log
I20251027 02:49:53.373631 1734687 ts_tablet_manager.cc:1403] T e347abcda9de4f348c443d4c39b2d7de P 29af00c3fa84488f95905b28458dbbeb: Time spent bootstrapping tablet: real 0.064s user 0.019s sys 0.009s
I20251027 02:49:53.378569 1734687 raft_consensus.cc:359] T e347abcda9de4f348c443d4c39b2d7de P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.379215 1734687 raft_consensus.cc:385] T e347abcda9de4f348c443d4c39b2d7de P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:49:53.379088 1734691 raft_consensus.cc:515] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.379463 1734687 raft_consensus.cc:740] T e347abcda9de4f348c443d4c39b2d7de P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29af00c3fa84488f95905b28458dbbeb, State: Initialized, Role: FOLLOWER
I20251027 02:49:53.380642 1734691 leader_election.cc:290] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [CANDIDATE]: Term 1 election: Requested vote from peers 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911), c3f7def5fcba459bb5e0469d9ea5b690 (127.152.231.194:36183)
I20251027 02:49:53.380604 1734687 consensus_queue.cc:260] T e347abcda9de4f348c443d4c39b2d7de P 29af00c3fa84488f95905b28458dbbeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.382314 1734192 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f3c4dd053dff44c28a4352a07dc4325e" candidate_uuid: "2a33e47daa87477b862ff20773137d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3f7def5fcba459bb5e0469d9ea5b690"
I20251027 02:49:53.382623 1734687 ts_tablet_manager.cc:1434] T e347abcda9de4f348c443d4c39b2d7de P 29af00c3fa84488f95905b28458dbbeb: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20251027 02:49:53.382889 1734192 raft_consensus.cc:3060] T f3c4dd053dff44c28a4352a07dc4325e P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:53.389601 1734192 raft_consensus.cc:2468] T f3c4dd053dff44c28a4352a07dc4325e P c3f7def5fcba459bb5e0469d9ea5b690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a33e47daa87477b862ff20773137d55 in term 1.
I20251027 02:49:53.390734 1734401 leader_election.cc:304] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a33e47daa87477b862ff20773137d55, c3f7def5fcba459bb5e0469d9ea5b690; no voters:
I20251027 02:49:53.391511 1734691 raft_consensus.cc:2804] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:49:53.391934 1734691 raft_consensus.cc:697] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [term 1 LEADER]: Becoming Leader. State: Replica: 2a33e47daa87477b862ff20773137d55, State: Running, Role: LEADER
I20251027 02:49:53.392645 1734691 consensus_queue.cc:237] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.396260 1733832 catalog_manager.cc:5649] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a33e47daa87477b862ff20773137d55 (127.152.231.195). New cstate: current_term: 1 leader_uuid: "2a33e47daa87477b862ff20773137d55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } health_report { overall_health: HEALTHY } } }
I20251027 02:49:53.412511 1733833 catalog_manager.cc:5649] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3f7def5fcba459bb5e0469d9ea5b690 (127.152.231.194). New cstate: current_term: 1 leader_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } health_report { overall_health: UNKNOWN } } }
I20251027 02:49:53.426126 1733832 catalog_manager.cc:5649] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a33e47daa87477b862ff20773137d55 (127.152.231.195). New cstate: current_term: 1 leader_uuid: "2a33e47daa87477b862ff20773137d55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } health_report { overall_health: HEALTHY } } }
I20251027 02:49:53.432062 1734467 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2dfe0d3741334c98b3615811f8a993d7" candidate_uuid: "29af00c3fa84488f95905b28458dbbeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a33e47daa87477b862ff20773137d55" is_pre_election: true
I20251027 02:49:53.432658 1734467 raft_consensus.cc:2468] T 2dfe0d3741334c98b3615811f8a993d7 P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29af00c3fa84488f95905b28458dbbeb in term 0.
I20251027 02:49:53.433984 1733985 leader_election.cc:304] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29af00c3fa84488f95905b28458dbbeb, 2a33e47daa87477b862ff20773137d55; no voters:
I20251027 02:49:53.364090 1734048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f3c4dd053dff44c28a4352a07dc4325e" candidate_uuid: "2a33e47daa87477b862ff20773137d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29af00c3fa84488f95905b28458dbbeb" is_pre_election: true
I20251027 02:49:53.447479 1734048 raft_consensus.cc:2468] T f3c4dd053dff44c28a4352a07dc4325e P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a33e47daa87477b862ff20773137d55 in term 0.
I20251027 02:49:53.448508 1734692 raft_consensus.cc:493] T a3f6339512a5402b96c0ded4bc9ed17a P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:49:53.449002 1734692 raft_consensus.cc:515] T a3f6339512a5402b96c0ded4bc9ed17a P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.450882 1734692 leader_election.cc:290] T a3f6339512a5402b96c0ded4bc9ed17a P c3f7def5fcba459bb5e0469d9ea5b690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911), 2a33e47daa87477b862ff20773137d55 (127.152.231.195:33863)
I20251027 02:49:53.451944 1734467 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3f6339512a5402b96c0ded4bc9ed17a" candidate_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a33e47daa87477b862ff20773137d55" is_pre_election: true
I20251027 02:49:53.451910 1734048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a3f6339512a5402b96c0ded4bc9ed17a" candidate_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29af00c3fa84488f95905b28458dbbeb" is_pre_election: true
I20251027 02:49:53.452687 1734467 raft_consensus.cc:2393] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c3f7def5fcba459bb5e0469d9ea5b690 in current term 1: Already voted for candidate 2a33e47daa87477b862ff20773137d55 in this term.
I20251027 02:49:53.453630 1734127 leader_election.cc:304] T a3f6339512a5402b96c0ded4bc9ed17a P c3f7def5fcba459bb5e0469d9ea5b690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3f7def5fcba459bb5e0469d9ea5b690; no voters: 29af00c3fa84488f95905b28458dbbeb, 2a33e47daa87477b862ff20773137d55
I20251027 02:49:53.454980 1734692 raft_consensus.cc:3060] T a3f6339512a5402b96c0ded4bc9ed17a P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:53.461517 1734048 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f3c4dd053dff44c28a4352a07dc4325e" candidate_uuid: "2a33e47daa87477b862ff20773137d55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29af00c3fa84488f95905b28458dbbeb"
I20251027 02:49:53.462182 1734048 raft_consensus.cc:3060] T f3c4dd053dff44c28a4352a07dc4325e P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:53.469851 1734048 raft_consensus.cc:2468] T f3c4dd053dff44c28a4352a07dc4325e P 29af00c3fa84488f95905b28458dbbeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a33e47daa87477b862ff20773137d55 in term 1.
I20251027 02:49:53.470806 1734692 raft_consensus.cc:2749] T a3f6339512a5402b96c0ded4bc9ed17a P c3f7def5fcba459bb5e0469d9ea5b690 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251027 02:49:53.475078 1734049 raft_consensus.cc:2468] T a3f6339512a5402b96c0ded4bc9ed17a P 29af00c3fa84488f95905b28458dbbeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a33e47daa87477b862ff20773137d55 in term 1.
I20251027 02:49:53.475960 1734192 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2dfe0d3741334c98b3615811f8a993d7" candidate_uuid: "29af00c3fa84488f95905b28458dbbeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" is_pre_election: true
I20251027 02:49:53.476631 1734192 raft_consensus.cc:2468] T 2dfe0d3741334c98b3615811f8a993d7 P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 29af00c3fa84488f95905b28458dbbeb in term 0.
I20251027 02:49:53.477794 1734401 leader_election.cc:304] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 29af00c3fa84488f95905b28458dbbeb, 2a33e47daa87477b862ff20773137d55; no voters: c3f7def5fcba459bb5e0469d9ea5b690
I20251027 02:49:53.478680 1734691 raft_consensus.cc:2804] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:49:53.479192 1734691 raft_consensus.cc:697] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [term 1 LEADER]: Becoming Leader. State: Replica: 2a33e47daa87477b862ff20773137d55, State: Running, Role: LEADER
I20251027 02:49:53.480053 1734691 consensus_queue.cc:237] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.498060 1734693 raft_consensus.cc:2804] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:49:53.498502 1734693 raft_consensus.cc:493] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:49:53.498878 1734693 raft_consensus.cc:3060] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:53.506781 1734693 raft_consensus.cc:515] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.507381 1733832 catalog_manager.cc:5649] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a33e47daa87477b862ff20773137d55 (127.152.231.195). New cstate: current_term: 1 leader_uuid: "2a33e47daa87477b862ff20773137d55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } health_report { overall_health: HEALTHY } } }
I20251027 02:49:53.508747 1734693 leader_election.cc:290] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [CANDIDATE]: Term 1 election: Requested vote from peers c3f7def5fcba459bb5e0469d9ea5b690 (127.152.231.194:36183), 2a33e47daa87477b862ff20773137d55 (127.152.231.195:33863)
I20251027 02:49:53.509562 1734192 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2dfe0d3741334c98b3615811f8a993d7" candidate_uuid: "29af00c3fa84488f95905b28458dbbeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3f7def5fcba459bb5e0469d9ea5b690"
I20251027 02:49:53.509722 1734467 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2dfe0d3741334c98b3615811f8a993d7" candidate_uuid: "29af00c3fa84488f95905b28458dbbeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a33e47daa87477b862ff20773137d55"
I20251027 02:49:53.510100 1734192 raft_consensus.cc:3060] T 2dfe0d3741334c98b3615811f8a993d7 P c3f7def5fcba459bb5e0469d9ea5b690 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:53.510250 1734467 raft_consensus.cc:3060] T 2dfe0d3741334c98b3615811f8a993d7 P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:53.515754 1734192 raft_consensus.cc:2468] T 2dfe0d3741334c98b3615811f8a993d7 P c3f7def5fcba459bb5e0469d9ea5b690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29af00c3fa84488f95905b28458dbbeb in term 1.
I20251027 02:49:53.516615 1734467 raft_consensus.cc:2468] T 2dfe0d3741334c98b3615811f8a993d7 P 2a33e47daa87477b862ff20773137d55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 29af00c3fa84488f95905b28458dbbeb in term 1.
I20251027 02:49:53.516810 1733984 leader_election.cc:304] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29af00c3fa84488f95905b28458dbbeb, c3f7def5fcba459bb5e0469d9ea5b690; no voters:
I20251027 02:49:53.518208 1734693 raft_consensus.cc:2804] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:49:53.533252 1734693 raft_consensus.cc:697] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [term 1 LEADER]: Becoming Leader. State: Replica: 29af00c3fa84488f95905b28458dbbeb, State: Running, Role: LEADER
I20251027 02:49:53.534312 1734693 consensus_queue.cc:237] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } }
I20251027 02:49:53.576195 1733832 catalog_manager.cc:5649] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb reported cstate change: term changed from 0 to 1, leader changed from <none> to 29af00c3fa84488f95905b28458dbbeb (127.152.231.193). New cstate: current_term: 1 leader_uuid: "29af00c3fa84488f95905b28458dbbeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 } health_report { overall_health: UNKNOWN } } }
I20251027 02:49:53.703465 1734049 raft_consensus.cc:3060] T e347abcda9de4f348c443d4c39b2d7de P 29af00c3fa84488f95905b28458dbbeb [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:53.712559 1734692 consensus_queue.cc:1048] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:49:53.762416 1734704 consensus_queue.cc:1048] T e347abcda9de4f348c443d4c39b2d7de P c3f7def5fcba459bb5e0469d9ea5b690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:49:53.764602 1734467 raft_consensus.cc:3060] T a7d21b4a0a0f4809822b7717e74a0710 P 2a33e47daa87477b862ff20773137d55 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:49:53.772046 1734704 consensus_queue.cc:1048] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:49:53.808833 1734707 consensus_queue.cc:1048] T a7d21b4a0a0f4809822b7717e74a0710 P c3f7def5fcba459bb5e0469d9ea5b690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251027 02:49:53.878614 1734691 consensus_queue.cc:1048] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:49:53.903028 1734691 consensus_queue.cc:1048] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251027 02:49:53.923163 1734713 consensus_queue.cc:1048] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20251027 02:49:53.932401 1734693 consensus_queue.cc:1048] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a33e47daa87477b862ff20773137d55" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 33863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:49:53.946738 1734193 raft_consensus.cc:1275] T 2dfe0d3741334c98b3615811f8a993d7 P c3f7def5fcba459bb5e0469d9ea5b690 [term 1 FOLLOWER]: Refusing update from remote peer 29af00c3fa84488f95905b28458dbbeb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:49:53.947878 1734049 raft_consensus.cc:1275] T f3c4dd053dff44c28a4352a07dc4325e P 29af00c3fa84488f95905b28458dbbeb [term 1 FOLLOWER]: Refusing update from remote peer 2a33e47daa87477b862ff20773137d55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:49:53.948158 1734693 consensus_queue.cc:1048] T 2dfe0d3741334c98b3615811f8a993d7 P 29af00c3fa84488f95905b28458dbbeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:49:53.954137 1734049 raft_consensus.cc:1275] T a3f6339512a5402b96c0ded4bc9ed17a P 29af00c3fa84488f95905b28458dbbeb [term 1 FOLLOWER]: Refusing update from remote peer 2a33e47daa87477b862ff20773137d55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:49:53.955299 1734691 consensus_queue.cc:1048] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:49:53.951774 1734765 consensus_queue.cc:1048] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29af00c3fa84488f95905b28458dbbeb" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 43911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20251027 02:49:54.000173 1734192 raft_consensus.cc:1275] T a3f6339512a5402b96c0ded4bc9ed17a P c3f7def5fcba459bb5e0469d9ea5b690 [term 1 FOLLOWER]: Refusing update from remote peer 2a33e47daa87477b862ff20773137d55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:49:54.010346 1734765 consensus_queue.cc:1048] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3f7def5fcba459bb5e0469d9ea5b690" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 36183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251027 02:49:54.410892 1734091 debug-util.cc:398] Leaking SignalData structure 0x7b08000ba980 after lost signal to thread 1733935
W20251027 02:49:58.966601 1734819 debug-util.cc:398] Leaking SignalData structure 0x7b080009c0c0 after lost signal to thread 1733605
W20251027 02:49:58.967391 1734819 debug-util.cc:398] Leaking SignalData structure 0x7b08000a7b80 after lost signal to thread 1733688
W20251027 02:50:03.017552 1733685 debug-util.cc:398] Leaking SignalData structure 0x7b08000aa6e0 after lost signal to thread 1733605
W20251027 02:50:03.018308 1733685 debug-util.cc:398] Leaking SignalData structure 0x7b080009f6c0 after lost signal to thread 1733688
W20251027 02:50:08.210458 1734946 debug-util.cc:398] Leaking SignalData structure 0x7b0800095c20 after lost signal to thread 1733697
W20251027 02:50:08.211208 1734946 debug-util.cc:398] Leaking SignalData structure 0x7b08000ace60 after lost signal to thread 1733783
E20251027 02:50:14.077979 1735118 webserver.cc:578] Webserver: sslize failed:
E20251027 02:50:14.078542 1735118 webserver.cc:578] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/webserver-crawl-itest.cc:304: Failure
Value of: s.IsRemoteError()
Actual: false
Expected: true
Network error: curl error: Server returned nothing (no headers, no data): Empty reply from server
Google Test trace:
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/webserver-crawl-itest.cc:301: https://127.152.231.252:44915/config
I20251027 02:50:14.101557 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1733912
W20251027 02:50:14.191879 1734401 connection.cc:537] client connection to 127.152.231.193:43911 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251027 02:50:14.192437 1734401 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251027 02:50:14.200042 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1734105
W20251027 02:50:14.219699 1734401 consensus_peers.cc:597] T f3c4dd053dff44c28a4352a07dc4325e P 2a33e47daa87477b862ff20773137d55 -> Peer 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911): Couldn't send request to peer 29af00c3fa84488f95905b28458dbbeb. Status: Network error: Client connection negotiation failed: client connection to 127.152.231.193:43911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251027 02:50:14.236891 1734401 consensus_peers.cc:597] T 198ae238c5074f5390d31f3ca289c24d P 2a33e47daa87477b862ff20773137d55 -> Peer 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911): Couldn't send request to peer 29af00c3fa84488f95905b28458dbbeb. Status: Network error: Client connection negotiation failed: client connection to 127.152.231.193:43911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251027 02:50:14.243438 1734401 consensus_peers.cc:597] T a3f6339512a5402b96c0ded4bc9ed17a P 2a33e47daa87477b862ff20773137d55 -> Peer 29af00c3fa84488f95905b28458dbbeb (127.152.231.193:43911): Couldn't send request to peer 29af00c3fa84488f95905b28458dbbeb. Status: Network error: Client connection negotiation failed: client connection to 127.152.231.193:43911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251027 02:50:14.297653 1734401 connection.cc:537] client connection to 127.152.231.194:36183 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251027 02:50:14.298281 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1734258
I20251027 02:50:14.389307 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1733604
I20251027 02:50:14.471006 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1733691
W20251027 02:50:14.471195 1733817 connection.cc:537] client connection to 127.152.231.254:38201 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251027 02:50:14.471658 1733817 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251027 02:50:14.515133 1733817 consensus_peers.cc:597] T 00000000000000000000000000000000 P 0b4ee841b8324bc5bec17338b9abf918 -> Peer 754fb6432e6f4fc89bfe76c08015f9c3 (127.152.231.254:38201): Couldn't send request to peer 754fb6432e6f4fc89bfe76c08015f9c3. Status: Network error: Client connection negotiation failed: client connection to 127.152.231.254:38201: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251027 02:50:14.561990 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1733786
2025-10-27T02:50:14Z chronyd exiting
I20251027 02:50:14.662972 1729439 test_util.cc:183] -----------------------------------------------
I20251027 02:50:14.663309 1729439 test_util.cc:184] Had failures, leaving test files at /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_2.1761533243417696-1729439-0
[ FAILED ] WebserverCrawlITest.TestAllWebPages/2, where GetParam() = (UseSsl::YES, ImpersonateKnox::NO) (35396 ms)
[ RUN ] WebserverCrawlITest.TestAllWebPages/3
I20251027 02:50:14.690375 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
2025-10-27T02:50:14Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-10-27T02:50:14Z Disabled control of system clock
I20251027 02:50:14.748443 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:50:14.749007 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:50:14.749418 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:50:14.752439 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.254:43929
--webserver_interface=127.152.231.254
--webserver_port=0
--builtin_ntp_servers=127.152.231.212:33957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.152.231.254:43929,127.152.231.253:46719,127.152.231.252:35507
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.cert
--webserver_private_key_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.key
--webserver_private_key_password_cmd=echo test with env {}
W20251027 02:50:15.355690 1735143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:50:15.356632 1735143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:50:15.357714 1735143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:50:15.416755 1735143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251027 02:50:15.417323 1735143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:50:15.417738 1735143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251027 02:50:15.418169 1735143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251027 02:50:15.535719 1735143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:33957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.152.231.254:43929,127.152.231.253:46719,127.152.231.252:35507
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.254:43929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-0/data/info.pb
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.cert
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.254
--webserver_port=0
--webserver_private_key_file=<redacted>
--webserver_private_key_password_cmd=<redacted>
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:50:15.537724 1735143 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:50:15.540191 1735143 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:50:15.593034 1735166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:50:16.964345 1735165 debug-util.cc:398] Leaking SignalData structure 0x7b0800034a00 after lost signal to thread 1735143
W20251027 02:50:17.206704 1735143 thread.cc:641] GCE (cloud detector) Time spent creating pthread: real 1.597s user 0.002s sys 0.001s
W20251027 02:50:17.207118 1735143 thread.cc:608] GCE (cloud detector) Time spent starting thread: real 1.598s user 0.002s sys 0.001s
W20251027 02:50:15.603907 1735167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:50:17.228076 1735214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:50:17.237715 1735143 server_base.cc:1047] running on GCE node
I20251027 02:50:17.239336 1735143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:50:17.261792 1735143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:50:17.263326 1735143 hybrid_clock.cc:648] HybridClock initialized: now 1761533417263272 us; error 42 us; skew 500 ppm
I20251027 02:50:17.264532 1735143 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:50:17.349633 1735143 webserver.cc:492] Webserver started at https://127.152.231.254:41179/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:50:17.351302 1735143 fs_manager.cc:362] Metadata directory not provided
I20251027 02:50:17.351647 1735143 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:50:17.352404 1735143 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:50:17.360260 1735143 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-0/data/instance:
uuid: "6e5a584fe76044d094e8a6a468189573"
format_stamp: "Formatted at 2025-10-27 02:50:17 on 5fd53c4cbb9d"
I20251027 02:50:17.362252 1735143 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-0/wal/instance:
uuid: "6e5a584fe76044d094e8a6a468189573"
format_stamp: "Formatted at 2025-10-27 02:50:17 on 5fd53c4cbb9d"
I20251027 02:50:17.401945 1735143 fs_manager.cc:696] Time spent creating directory manager: real 0.039s user 0.011s sys 0.012s
I20251027 02:50:17.418382 1735222 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:50:17.434759 1735143 fs_manager.cc:730] Time spent opening block manager: real 0.028s user 0.004s sys 0.004s
I20251027 02:50:17.435293 1735143 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-0/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-0/wal
uuid: "6e5a584fe76044d094e8a6a468189573"
format_stamp: "Formatted at 2025-10-27 02:50:17 on 5fd53c4cbb9d"
I20251027 02:50:17.435745 1735143 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-0/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-0/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:50:17.658449 1735143 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:50:17.660442 1735143 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:50:17.661175 1735143 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:50:18.513995 1735295 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.254:43929 every 8 connection(s)
I20251027 02:50:18.521426 1735143 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.254:43929
I20251027 02:50:18.525508 1735143 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-0/data/info.pb
I20251027 02:50:18.534003 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1735143
I20251027 02:50:18.534673 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-0/wal/instance
I20251027 02:50:18.539075 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.253:46719
--webserver_interface=127.152.231.253
--webserver_port=0
--builtin_ntp_servers=127.152.231.212:33957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.152.231.254:43929,127.152.231.253:46719,127.152.231.252:35507
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.cert
--webserver_private_key_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.key
--webserver_private_key_password_cmd=echo test with env {}
I20251027 02:50:18.548307 1735296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:18.596177 1735296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:50:18.664908 1735296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } has no permanent_uuid. Determining permanent_uuid...
W20251027 02:50:18.668048 1735230 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.152.231.253:46719: connect: Connection refused (error 111)
W20251027 02:50:18.688553 1735296 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:46719: Network error: Client connection negotiation failed: client connection to 127.152.231.253:46719: connect: Connection refused (error 111)
I20251027 02:50:18.738179 1735296 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } attempt: 1
W20251027 02:50:18.744477 1735296 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:46719: Network error: Client connection negotiation failed: client connection to 127.152.231.253:46719: connect: Connection refused (error 111)
I20251027 02:50:18.812950 1735296 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } attempt: 2
W20251027 02:50:18.818368 1735296 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:46719: Network error: Client connection negotiation failed: client connection to 127.152.231.253:46719: connect: Connection refused (error 111)
I20251027 02:50:18.909818 1735296 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } attempt: 3
W20251027 02:50:18.915232 1735296 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:46719: Network error: Client connection negotiation failed: client connection to 127.152.231.253:46719: connect: Connection refused (error 111)
I20251027 02:50:19.058669 1735296 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } attempt: 4
W20251027 02:50:19.063643 1735296 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:46719: Network error: Client connection negotiation failed: client connection to 127.152.231.253:46719: connect: Connection refused (error 111)
W20251027 02:50:19.338672 1735298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:50:19.339524 1735298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:50:19.340426 1735298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20251027 02:50:19.363075 1735296 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } attempt: 5
W20251027 02:50:19.368718 1735296 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:46719: Network error: Client connection negotiation failed: client connection to 127.152.231.253:46719: connect: Connection refused (error 111)
W20251027 02:50:19.397260 1735298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251027 02:50:19.397785 1735298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:50:19.398190 1735298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251027 02:50:19.398597 1735298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251027 02:50:19.460402 1735298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:33957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.152.231.254:43929,127.152.231.253:46719,127.152.231.252:35507
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.253:46719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-1/data/info.pb
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.cert
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.253
--webserver_port=0
--webserver_private_key_file=<redacted>
--webserver_private_key_password_cmd=<redacted>
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:50:19.462344 1735298 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:50:19.464708 1735298 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:50:19.502017 1735308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:50:19.916126 1735296 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } attempt: 6
W20251027 02:50:19.955178 1735296 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:46719: Network error: Client connection negotiation failed: client connection to 127.152.231.253:46719: connect: Connection refused (error 111)
I20251027 02:50:21.015648 1735296 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } attempt: 7
W20251027 02:50:21.048352 1735296 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.253:46719: Network error: Client connection negotiation failed: client connection to 127.152.231.253:46719: connect: Connection refused (error 111)
W20251027 02:50:20.886420 1735306 debug-util.cc:398] Leaking SignalData structure 0x7b0800034a00 after lost signal to thread 1735298
W20251027 02:50:21.214941 1735298 thread.cc:641] GCE (cloud detector) Time spent creating pthread: real 1.705s user 0.605s sys 0.846s
W20251027 02:50:21.217774 1735298 thread.cc:608] GCE (cloud detector) Time spent starting thread: real 1.708s user 0.605s sys 0.847s
W20251027 02:50:19.523319 1735309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:50:21.238330 1735298 server_base.cc:1047] running on GCE node
W20251027 02:50:21.246374 1735329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:50:21.248111 1735298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:50:21.265352 1735298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:50:21.266896 1735298 hybrid_clock.cc:648] HybridClock initialized: now 1761533421266799 us; error 86 us; skew 500 ppm
I20251027 02:50:21.268054 1735298 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:50:21.362074 1735298 webserver.cc:492] Webserver started at https://127.152.231.253:33307/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:50:21.363837 1735298 fs_manager.cc:362] Metadata directory not provided
I20251027 02:50:21.364266 1735298 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:50:21.365051 1735298 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:50:21.373528 1735298 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-1/data/instance:
uuid: "58b24254082d494d8493113ac2cf0bd0"
format_stamp: "Formatted at 2025-10-27 02:50:21 on 5fd53c4cbb9d"
I20251027 02:50:21.390650 1735298 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-1/wal/instance:
uuid: "58b24254082d494d8493113ac2cf0bd0"
format_stamp: "Formatted at 2025-10-27 02:50:21 on 5fd53c4cbb9d"
I20251027 02:50:21.414011 1735298 fs_manager.cc:696] Time spent creating directory manager: real 0.022s user 0.005s sys 0.004s
I20251027 02:50:21.446345 1735335 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:50:21.462155 1735298 fs_manager.cc:730] Time spent opening block manager: real 0.043s user 0.007s sys 0.003s
I20251027 02:50:21.462675 1735298 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-1/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-1/wal
uuid: "58b24254082d494d8493113ac2cf0bd0"
format_stamp: "Formatted at 2025-10-27 02:50:21 on 5fd53c4cbb9d"
I20251027 02:50:21.463162 1735298 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-1/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-1/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:50:21.640761 1735298 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:50:21.642750 1735298 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:50:21.643468 1735298 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:50:22.477914 1735298 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.253:46719
I20251027 02:50:22.478082 1735429 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.253:46719 every 8 connection(s)
I20251027 02:50:22.482067 1735298 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-1/data/info.pb
I20251027 02:50:22.488071 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1735298
I20251027 02:50:22.488619 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-1/wal/instance
I20251027 02:50:22.492074 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.252:35507
--webserver_interface=127.152.231.252
--webserver_port=0
--builtin_ntp_servers=127.152.231.212:33957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.152.231.254:43929,127.152.231.253:46719,127.152.231.252:35507
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.cert
--webserver_private_key_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.key
--webserver_private_key_password_cmd=echo test with env {}
I20251027 02:50:22.516024 1735430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:22.535347 1735430 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:50:22.596637 1735430 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:50:22.644551 1735430 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } has no permanent_uuid. Determining permanent_uuid...
W20251027 02:50:22.648020 1735342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111)
W20251027 02:50:22.679045 1735430 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:35507: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111)
I20251027 02:50:22.728638 1735430 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } attempt: 1
W20251027 02:50:22.732916 1735430 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:35507: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111)
I20251027 02:50:22.801390 1735430 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } attempt: 2
W20251027 02:50:22.805588 1735430 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:35507: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111)
I20251027 02:50:22.897011 1735430 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } attempt: 3
W20251027 02:50:22.901759 1735430 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:35507: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111)
I20251027 02:50:23.045171 1735430 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } attempt: 4
W20251027 02:50:23.050715 1735430 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:35507: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111)
W20251027 02:50:23.109790 1735432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:50:23.110752 1735432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:50:23.111794 1735432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20251027 02:50:23.138919 1735296 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } attempt: 8
W20251027 02:50:23.167932 1735432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251027 02:50:23.168485 1735432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:50:23.168887 1735432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251027 02:50:23.169309 1735432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251027 02:50:23.181165 1735296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } has no permanent_uuid. Determining permanent_uuid...
W20251027 02:50:23.199194 1735296 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:35507: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111)
I20251027 02:50:23.234297 1735432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:33957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.152.231.254:43929,127.152.231.253:46719,127.152.231.252:35507
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.152.231.252:35507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-2/data/info.pb
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.cert
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.252
--webserver_port=0
--webserver_private_key_file=<redacted>
--webserver_private_key_password_cmd=<redacted>
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:50:23.236429 1735432 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:50:23.239063 1735432 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251027 02:50:23.264649 1735296 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } attempt: 1
W20251027 02:50:23.269657 1735296 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:35507: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111)
W20251027 02:50:23.293036 1735461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:50:23.323058 1735296 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } attempt: 2
W20251027 02:50:23.327591 1735296 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:35507: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111)
I20251027 02:50:23.350170 1735430 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } attempt: 5
W20251027 02:50:23.355125 1735430 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:35507: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111)
I20251027 02:50:23.404055 1735296 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } attempt: 3
W20251027 02:50:23.409065 1735296 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:35507: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111)
I20251027 02:50:23.564534 1735296 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } attempt: 4
W20251027 02:50:23.569041 1735296 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:35507: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111)
I20251027 02:50:23.865506 1735296 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } attempt: 5
W20251027 02:50:23.867748 1735223 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111) [suppressed 12 similar messages]
W20251027 02:50:23.870540 1735296 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:35507: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111)
I20251027 02:50:23.902582 1735430 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } attempt: 6
W20251027 02:50:23.931536 1735430 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:35507: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111)
I20251027 02:50:24.391973 1735296 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } attempt: 6
W20251027 02:50:24.435118 1735296 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:35507: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111)
W20251027 02:50:24.663221 1735460 debug-util.cc:398] Leaking SignalData structure 0x7b0800034a00 after lost signal to thread 1735432
W20251027 02:50:23.307929 1735462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:50:24.964792 1735432 thread.cc:641] GCE (cloud detector) Time spent creating pthread: real 1.655s user 0.002s sys 0.001s
W20251027 02:50:24.965283 1735432 thread.cc:608] GCE (cloud detector) Time spent starting thread: real 1.656s user 0.002s sys 0.001s
W20251027 02:50:24.992058 1735512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:50:24.991991 1735430 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } attempt: 7
I20251027 02:50:24.993495 1735432 server_base.cc:1047] running on GCE node
I20251027 02:50:24.995043 1735432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:50:25.006204 1735432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:50:25.007740 1735432 hybrid_clock.cc:648] HybridClock initialized: now 1761533425007654 us; error 85 us; skew 500 ppm
I20251027 02:50:25.009004 1735432 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20251027 02:50:25.032179 1735430 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:35507: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111)
I20251027 02:50:25.076304 1735432 webserver.cc:492] Webserver started at https://127.152.231.252:46701/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:50:25.078075 1735432 fs_manager.cc:362] Metadata directory not provided
I20251027 02:50:25.078486 1735432 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:50:25.079262 1735432 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:50:25.087606 1735432 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-2/data/instance:
uuid: "61be957d7484418387fd600571cc9033"
format_stamp: "Formatted at 2025-10-27 02:50:25 on 5fd53c4cbb9d"
I20251027 02:50:25.089673 1735432 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-2/wal/instance:
uuid: "61be957d7484418387fd600571cc9033"
format_stamp: "Formatted at 2025-10-27 02:50:25 on 5fd53c4cbb9d"
I20251027 02:50:25.115645 1735432 fs_manager.cc:696] Time spent creating directory manager: real 0.025s user 0.016s sys 0.008s
I20251027 02:50:25.157558 1735526 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:50:25.159307 1735432 fs_manager.cc:730] Time spent opening block manager: real 0.039s user 0.011s sys 0.005s
I20251027 02:50:25.159826 1735432 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-2/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-2/wal
uuid: "61be957d7484418387fd600571cc9033"
format_stamp: "Formatted at 2025-10-27 02:50:25 on 5fd53c4cbb9d"
I20251027 02:50:25.160385 1735432 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-2/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-2/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:50:25.315284 1735432 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:50:25.317368 1735432 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:50:25.318130 1735432 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:50:25.473224 1735296 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } attempt: 7
W20251027 02:50:25.500043 1735296 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.152.231.252:35507: Network error: Client connection negotiation failed: client connection to 127.152.231.252:35507: connect: Connection refused (error 111)
I20251027 02:50:26.229898 1735432 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.252:35507
I20251027 02:50:26.230139 1735601 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.252:35507 every 8 connection(s)
I20251027 02:50:26.234200 1735432 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-2/data/info.pb
I20251027 02:50:26.242089 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1735432
I20251027 02:50:26.242916 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/master-2/wal/instance
I20251027 02:50:26.271979 1735602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:26.320919 1735602 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:50:26.354933 1735602 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:50:26.374728 1735602 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } has no permanent_uuid. Determining permanent_uuid...
I20251027 02:50:26.395596 1735602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033: Bootstrap starting.
I20251027 02:50:26.403703 1735602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:26.419325 1735602 log.cc:826] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033: Log is configured to *not* fsync() on all Append() calls
I20251027 02:50:26.437170 1735602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033: No bootstrap required, opened a new log
I20251027 02:50:26.474582 1735602 raft_consensus.cc:359] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } }
I20251027 02:50:26.475549 1735602 raft_consensus.cc:385] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:26.475872 1735602 raft_consensus.cc:740] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61be957d7484418387fd600571cc9033, State: Initialized, Role: FOLLOWER
I20251027 02:50:26.476889 1735602 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } }
I20251027 02:50:26.509016 1735602 sys_catalog.cc:565] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [sys.catalog]: configured and running, proceeding with master startup.
I20251027 02:50:26.506529 1735616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } } }
I20251027 02:50:26.524149 1735616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [sys.catalog]: This master's current role is: FOLLOWER
W20251027 02:50:26.621975 1735533 tablet.cc:2378] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:50:26.710534 1735634 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20251027 02:50:26.718508 1735616 raft_consensus.cc:493] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:50:26.719144 1735616 raft_consensus.cc:515] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } }
I20251027 02:50:26.722203 1735616 leader_election.cc:290] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e5a584fe76044d094e8a6a468189573 (127.152.231.254:43929), 58b24254082d494d8493113ac2cf0bd0 (127.152.231.253:46719)
I20251027 02:50:26.728996 1735432 master_runner.cc:186] Error getting master registration for 127.152.231.254:43929: OK, instance_id { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" instance_seqno: 1761533418253439 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20251027 02:50:26.780441 1735432 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20251027 02:50:27.122618 1735430 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } attempt: 8
I20251027 02:50:27.162822 1735430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0: Bootstrap starting.
I20251027 02:50:27.169951 1735430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:27.172094 1735430 log.cc:826] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0: Log is configured to *not* fsync() on all Append() calls
I20251027 02:50:27.204567 1735430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0: No bootstrap required, opened a new log
I20251027 02:50:27.248814 1735430 raft_consensus.cc:359] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } }
I20251027 02:50:27.265340 1735430 raft_consensus.cc:385] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:27.265787 1735430 raft_consensus.cc:740] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58b24254082d494d8493113ac2cf0bd0, State: Initialized, Role: FOLLOWER
I20251027 02:50:27.266875 1735430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } }
I20251027 02:50:27.278178 1735639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } } }
I20251027 02:50:27.287842 1735430 sys_catalog.cc:565] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0 [sys.catalog]: configured and running, proceeding with master startup.
I20251027 02:50:27.287999 1735388 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61be957d7484418387fd600571cc9033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58b24254082d494d8493113ac2cf0bd0" is_pre_election: true
I20251027 02:50:27.293298 1735639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:50:27.293727 1735388 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61be957d7484418387fd600571cc9033 in term 0.
I20251027 02:50:27.295121 1735531 leader_election.cc:304] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58b24254082d494d8493113ac2cf0bd0, 61be957d7484418387fd600571cc9033; no voters:
I20251027 02:50:27.305858 1735640 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:50:27.310580 1735640 raft_consensus.cc:493] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:50:27.311101 1735640 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:27.318650 1735640 raft_consensus.cc:515] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } }
I20251027 02:50:27.326436 1735640 leader_election.cc:290] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [CANDIDATE]: Term 1 election: Requested vote from peers 6e5a584fe76044d094e8a6a468189573 (127.152.231.254:43929), 58b24254082d494d8493113ac2cf0bd0 (127.152.231.253:46719)
I20251027 02:50:27.328088 1735388 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61be957d7484418387fd600571cc9033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58b24254082d494d8493113ac2cf0bd0"
I20251027 02:50:27.328649 1735388 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:27.334772 1735388 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61be957d7484418387fd600571cc9033 in term 1.
I20251027 02:50:27.335898 1735531 leader_election.cc:304] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58b24254082d494d8493113ac2cf0bd0, 61be957d7484418387fd600571cc9033; no voters:
I20251027 02:50:27.337608 1735640 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:50:27.353246 1735640 raft_consensus.cc:697] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [term 1 LEADER]: Becoming Leader. State: Replica: 61be957d7484418387fd600571cc9033, State: Running, Role: LEADER
I20251027 02:50:27.354360 1735640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } }
I20251027 02:50:27.379506 1735644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61be957d7484418387fd600571cc9033. Latest consensus state: current_term: 1 leader_uuid: "61be957d7484418387fd600571cc9033" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } } }
I20251027 02:50:27.380370 1735644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [sys.catalog]: This master's current role is: LEADER
I20251027 02:50:27.394862 1735648 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251027 02:50:27.429441 1735648 catalog_manager.cc:1494] Initializing Kudu cluster ID...
W20251027 02:50:27.446830 1735357 tablet.cc:2378] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:50:27.482452 1735388 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0 [term 1 FOLLOWER]: Refusing update from remote peer 61be957d7484418387fd600571cc9033: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:50:27.484144 1735644 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:50:27.547863 1735639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61be957d7484418387fd600571cc9033. Latest consensus state: current_term: 1 leader_uuid: "61be957d7484418387fd600571cc9033" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } } }
I20251027 02:50:27.548635 1735639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:50:27.574532 1735296 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } attempt: 8
I20251027 02:50:27.575891 1735644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "61be957d7484418387fd600571cc9033" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } } }
I20251027 02:50:27.576725 1735644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [sys.catalog]: This master's current role is: LEADER
W20251027 02:50:27.577706 1735656 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251027 02:50:27.578253 1735656 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251027 02:50:27.532768 1735298 master_runner.cc:186] Error getting master registration for 127.152.231.254:43929: OK, instance_id { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" instance_seqno: 1761533418253439 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20251027 02:50:27.590737 1735648 catalog_manager.cc:1357] Generated new cluster ID: bcf7bf33a4a54bff8ba35841bea953ae
I20251027 02:50:27.591095 1735648 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251027 02:50:27.594712 1735298 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20251027 02:50:27.594307 1735639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "61be957d7484418387fd600571cc9033" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } } }
I20251027 02:50:27.595062 1735639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:50:27.625349 1735296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573: Bootstrap starting.
I20251027 02:50:27.649613 1735296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:27.651798 1735296 log.cc:826] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573: Log is configured to *not* fsync() on all Append() calls
I20251027 02:50:27.662111 1735648 catalog_manager.cc:1380] Generated new certificate authority record
I20251027 02:50:27.664228 1735648 catalog_manager.cc:1514] Loading token signing keys...
I20251027 02:50:27.669858 1735296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573: No bootstrap required, opened a new log
I20251027 02:50:27.711925 1735648 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033: Generated new TSK 0
I20251027 02:50:27.713132 1735648 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251027 02:50:27.714156 1735296 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } }
I20251027 02:50:27.715044 1735296 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:27.715333 1735296 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e5a584fe76044d094e8a6a468189573, State: Initialized, Role: FOLLOWER
I20251027 02:50:27.742370 1735296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } }
I20251027 02:50:27.754254 1735665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } } }
I20251027 02:50:27.755335 1735665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:50:27.763127 1735296 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573 [sys.catalog]: configured and running, proceeding with master startup.
I20251027 02:50:27.763610 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:50:27.763242 1735258 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61be957d7484418387fd600571cc9033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e5a584fe76044d094e8a6a468189573"
I20251027 02:50:27.764478 1735256 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:27.764488 1735259 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61be957d7484418387fd600571cc9033" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e5a584fe76044d094e8a6a468189573" is_pre_election: true
I20251027 02:50:27.766202 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.152.231.193:0
--local_ip_for_outbound_sockets=127.152.231.193
--webserver_interface=127.152.231.193
--webserver_port=0
--tserver_master_addrs=127.152.231.254:43929,127.152.231.253:46719,127.152.231.252:35507
--builtin_ntp_servers=127.152.231.212:33957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.cert
--webserver_private_key_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.key
--webserver_private_key_password_cmd=echo test with env {}
I20251027 02:50:27.767048 1735259 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54550 (request call id 1) took 1044 ms. Trace:
I20251027 02:50:27.768056 1735259 rpcz_store.cc:276] 1027 02:50:26.723536 (+ 0us) service_pool.cc:168] Inserting onto call queue
1027 02:50:26.723681 (+ 145us) service_pool.cc:225] Handling call
1027 02:50:27.766961 (+1043280us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20251027 02:50:27.775777 1735256 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573 [term 1 FOLLOWER]: Refusing update from remote peer 61be957d7484418387fd600571cc9033: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251027 02:50:27.778046 1735644 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:50:27.935542 1735665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61be957d7484418387fd600571cc9033. Latest consensus state: current_term: 1 leader_uuid: "61be957d7484418387fd600571cc9033" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } } }
I20251027 02:50:27.936280 1735665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:50:27.937301 1735673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "61be957d7484418387fd600571cc9033" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } } }
I20251027 02:50:27.938812 1735673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573 [sys.catalog]: This master's current role is: FOLLOWER
I20251027 02:50:27.986095 1735682 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20251027 02:50:28.279656 1735644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "61be957d7484418387fd600571cc9033" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5a584fe76044d094e8a6a468189573" member_type: VOTER last_known_addr { host: "127.152.231.254" port: 43929 } } peers { permanent_uuid: "58b24254082d494d8493113ac2cf0bd0" member_type: VOTER last_known_addr { host: "127.152.231.253" port: 46719 } } peers { permanent_uuid: "61be957d7484418387fd600571cc9033" member_type: VOTER last_known_addr { host: "127.152.231.252" port: 35507 } } }
I20251027 02:50:28.280371 1735644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61be957d7484418387fd600571cc9033 [sys.catalog]: This master's current role is: LEADER
W20251027 02:50:28.487254 1735667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:50:28.488056 1735667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:50:28.489189 1735667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20251027 02:50:28.583250 1735656 catalog_manager.cc:1269] Loaded cluster ID: bcf7bf33a4a54bff8ba35841bea953ae
I20251027 02:50:28.583595 1735656 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0: loading cluster ID for follower catalog manager: success
W20251027 02:50:28.588939 1735667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
I20251027 02:50:28.590032 1735656 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0: acquiring CA information for follower catalog manager: success
W20251027 02:50:28.590698 1735667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.152.231.193
I20251027 02:50:28.595788 1735656 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 58b24254082d494d8493113ac2cf0bd0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251027 02:50:28.725921 1735667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:33957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.152.231.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-0/data/info.pb
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.cert
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.193
--webserver_port=0
--webserver_private_key_file=<redacted>
--webserver_private_key_password_cmd=<redacted>
--tserver_master_addrs=127.152.231.254:43929,127.152.231.253:46719,127.152.231.252:35507
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.152.231.193
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:50:28.742797 1735667 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:50:28.745448 1735667 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:50:28.792546 1735706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:50:29.009641 1735682 catalog_manager.cc:1269] Loaded cluster ID: bcf7bf33a4a54bff8ba35841bea953ae
I20251027 02:50:29.009959 1735682 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573: loading cluster ID for follower catalog manager: success
I20251027 02:50:29.028021 1735682 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573: acquiring CA information for follower catalog manager: success
I20251027 02:50:29.033519 1735682 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 6e5a584fe76044d094e8a6a468189573: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20251027 02:50:30.172902 1735704 debug-util.cc:398] Leaking SignalData structure 0x7b0800034920 after lost signal to thread 1735667
W20251027 02:50:30.431205 1735704 kernel_stack_watchdog.cc:198] Thread 1735667 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 365ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20251027 02:50:28.819032 1735707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:50:30.438285 1735667 thread.cc:641] GCE (cloud detector) Time spent creating pthread: real 1.633s user 0.603s sys 0.815s
W20251027 02:50:30.438669 1735667 thread.cc:608] GCE (cloud detector) Time spent starting thread: real 1.633s user 0.603s sys 0.815s
W20251027 02:50:30.475662 1735727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:50:30.489291 1735667 server_base.cc:1047] running on GCE node
I20251027 02:50:30.490845 1735667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:50:30.506075 1735667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:50:30.507639 1735667 hybrid_clock.cc:648] HybridClock initialized: now 1761533430507552 us; error 80 us; skew 500 ppm
I20251027 02:50:30.508908 1735667 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:50:30.618114 1735667 webserver.cc:492] Webserver started at https://127.152.231.193:40963/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:50:30.619810 1735667 fs_manager.cc:362] Metadata directory not provided
I20251027 02:50:30.620170 1735667 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:50:30.620862 1735667 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:50:30.628679 1735667 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-0/data/instance:
uuid: "ba7dd0e342744505993460bef2180d40"
format_stamp: "Formatted at 2025-10-27 02:50:30 on 5fd53c4cbb9d"
I20251027 02:50:30.642481 1735667 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-0/wal/instance:
uuid: "ba7dd0e342744505993460bef2180d40"
format_stamp: "Formatted at 2025-10-27 02:50:30 on 5fd53c4cbb9d"
I20251027 02:50:30.691052 1735667 fs_manager.cc:696] Time spent creating directory manager: real 0.048s user 0.009s sys 0.013s
I20251027 02:50:30.714133 1735734 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:50:30.715756 1735667 fs_manager.cc:730] Time spent opening block manager: real 0.020s user 0.009s sys 0.002s
I20251027 02:50:30.716208 1735667 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-0/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-0/wal
uuid: "ba7dd0e342744505993460bef2180d40"
format_stamp: "Formatted at 2025-10-27 02:50:30 on 5fd53c4cbb9d"
I20251027 02:50:30.716638 1735667 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-0/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-0/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:50:30.927544 1735667 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:50:30.929518 1735667 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:50:30.930397 1735667 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:50:30.945873 1735667 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251027 02:50:30.962554 1735667 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251027 02:50:30.962980 1735667 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.001s
I20251027 02:50:30.963431 1735667 ts_tablet_manager.cc:616] Registered 0 tablets
I20251027 02:50:30.963717 1735667 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251027 02:50:33.029803 1735667 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.193:40985
I20251027 02:50:33.030001 1735907 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.193:40985 every 8 connection(s)
I20251027 02:50:33.033511 1735667 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-0/data/info.pb
I20251027 02:50:33.042527 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1735667
I20251027 02:50:33.043095 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-0/wal/instance
I20251027 02:50:33.053481 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:50:33.056675 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.152.231.194:0
--local_ip_for_outbound_sockets=127.152.231.194
--webserver_interface=127.152.231.194
--webserver_port=0
--tserver_master_addrs=127.152.231.254:43929,127.152.231.253:46719,127.152.231.252:35507
--builtin_ntp_servers=127.152.231.212:33957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.cert
--webserver_private_key_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.key
--webserver_private_key_password_cmd=echo test with env {}
I20251027 02:50:33.127803 1735909 heartbeater.cc:344] Connected to a master server at 127.152.231.252:35507
I20251027 02:50:33.128386 1735909 heartbeater.cc:461] Registering TS with master...
I20251027 02:50:33.130005 1735909 heartbeater.cc:507] Master 127.152.231.252:35507 requested a full tablet report, sending...
I20251027 02:50:33.134231 1735559 ts_manager.cc:194] Registered new tserver with Master: ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985)
I20251027 02:50:33.137234 1735559 master_service.cc:502] Signed X509 certificate for tserver {username='jenkins-slave'} at 127.152.231.193:51543
I20251027 02:50:33.143569 1735914 heartbeater.cc:344] Connected to a master server at 127.152.231.253:46719
I20251027 02:50:33.143694 1735924 heartbeater.cc:344] Connected to a master server at 127.152.231.254:43929
I20251027 02:50:33.144012 1735914 heartbeater.cc:461] Registering TS with master...
I20251027 02:50:33.144060 1735924 heartbeater.cc:461] Registering TS with master...
I20251027 02:50:33.145031 1735914 heartbeater.cc:507] Master 127.152.231.253:46719 requested a full tablet report, sending...
I20251027 02:50:33.145068 1735924 heartbeater.cc:507] Master 127.152.231.254:43929 requested a full tablet report, sending...
I20251027 02:50:33.148458 1735369 ts_manager.cc:194] Registered new tserver with Master: ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985)
I20251027 02:50:33.148607 1735247 ts_manager.cc:194] Registered new tserver with Master: ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985)
W20251027 02:50:33.677455 1735920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:50:33.678261 1735920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:50:33.679363 1735920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:50:33.739315 1735920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:50:33.741447 1735920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.152.231.194
I20251027 02:50:33.812903 1735920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:33957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.152.231.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-1/data/info.pb
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.cert
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.194
--webserver_port=0
--webserver_private_key_file=<redacted>
--webserver_private_key_password_cmd=<redacted>
--tserver_master_addrs=127.152.231.254:43929,127.152.231.253:46719,127.152.231.252:35507
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.152.231.194
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:50:33.814926 1735920 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:50:33.817406 1735920 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:50:33.852434 1735973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:50:34.142091 1735909 heartbeater.cc:499] Master 127.152.231.252:35507 was elected leader, sending a full tablet report...
W20251027 02:50:35.244141 1735969 debug-util.cc:398] Leaking SignalData structure 0x7b0800034920 after lost signal to thread 1735920
W20251027 02:50:35.466389 1735969 kernel_stack_watchdog.cc:198] Thread 1735920 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 390ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20251027 02:50:35.467453 1735920 thread.cc:641] GCE (cloud detector) Time spent creating pthread: real 1.616s user 0.632s sys 0.888s
W20251027 02:50:35.467828 1735920 thread.cc:608] GCE (cloud detector) Time spent starting thread: real 1.616s user 0.632s sys 0.888s
W20251027 02:50:33.857674 1735975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:50:35.504055 1736045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:50:35.507659 1735920 server_base.cc:1047] running on GCE node
I20251027 02:50:35.509414 1735920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:50:35.533704 1735920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:50:35.535245 1735920 hybrid_clock.cc:648] HybridClock initialized: now 1761533435535143 us; error 91 us; skew 500 ppm
I20251027 02:50:35.536506 1735920 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:50:35.657657 1735920 webserver.cc:492] Webserver started at https://127.152.231.194:40289/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:50:35.659561 1735920 fs_manager.cc:362] Metadata directory not provided
I20251027 02:50:35.659974 1735920 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:50:35.660765 1735920 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:50:35.670176 1735920 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-1/data/instance:
uuid: "06b6ed712e4b46fa8519a5cfde23ccb9"
format_stamp: "Formatted at 2025-10-27 02:50:35 on 5fd53c4cbb9d"
I20251027 02:50:35.672122 1735920 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-1/wal/instance:
uuid: "06b6ed712e4b46fa8519a5cfde23ccb9"
format_stamp: "Formatted at 2025-10-27 02:50:35 on 5fd53c4cbb9d"
I20251027 02:50:35.719595 1735920 fs_manager.cc:696] Time spent creating directory manager: real 0.046s user 0.016s sys 0.000s
I20251027 02:50:35.750147 1736052 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:50:35.758442 1735920 fs_manager.cc:730] Time spent opening block manager: real 0.034s user 0.009s sys 0.007s
I20251027 02:50:35.758939 1735920 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-1/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-1/wal
uuid: "06b6ed712e4b46fa8519a5cfde23ccb9"
format_stamp: "Formatted at 2025-10-27 02:50:35 on 5fd53c4cbb9d"
I20251027 02:50:35.759414 1735920 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-1/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-1/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:50:36.018971 1735920 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:50:36.034942 1735920 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:50:36.035723 1735920 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:50:36.040383 1735920 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251027 02:50:36.048962 1735920 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251027 02:50:36.049355 1735920 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20251027 02:50:36.049700 1735920 ts_tablet_manager.cc:616] Registered 0 tablets
I20251027 02:50:36.049940 1735920 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251027 02:50:37.997819 1735920 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.194:38035
I20251027 02:50:37.998027 1736212 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.194:38035 every 8 connection(s)
I20251027 02:50:38.001546 1735920 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-1/data/info.pb
I20251027 02:50:38.005530 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1735920
I20251027 02:50:38.006381 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-1/wal/instance
I20251027 02:50:38.015861 1729439 net_util.cc:842] PID 1729439 is more than 18 bits wide, substituted with 156575
I20251027 02:50:38.018524 1729439 external_mini_cluster.cc:1366] Running /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.152.231.195:0
--local_ip_for_outbound_sockets=127.152.231.195
--webserver_interface=127.152.231.195
--webserver_port=0
--tserver_master_addrs=127.152.231.254:43929,127.152.231.253:46719,127.152.231.252:35507
--builtin_ntp_servers=127.152.231.212:33957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.cert
--webserver_private_key_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.key
--webserver_private_key_password_cmd=echo test with env {}
I20251027 02:50:38.104130 1736213 heartbeater.cc:344] Connected to a master server at 127.152.231.252:35507
I20251027 02:50:38.108595 1736213 heartbeater.cc:461] Registering TS with master...
I20251027 02:50:38.109529 1736214 heartbeater.cc:344] Connected to a master server at 127.152.231.253:46719
I20251027 02:50:38.109897 1736214 heartbeater.cc:461] Registering TS with master...
I20251027 02:50:38.110440 1736213 heartbeater.cc:507] Master 127.152.231.252:35507 requested a full tablet report, sending...
I20251027 02:50:38.111163 1736214 heartbeater.cc:507] Master 127.152.231.253:46719 requested a full tablet report, sending...
I20251027 02:50:38.113866 1735369 ts_manager.cc:194] Registered new tserver with Master: 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035)
I20251027 02:50:38.114212 1735559 ts_manager.cc:194] Registered new tserver with Master: 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035)
I20251027 02:50:38.116726 1735559 master_service.cc:502] Signed X509 certificate for tserver {username='jenkins-slave'} at 127.152.231.194:57675
I20251027 02:50:38.124485 1736218 heartbeater.cc:344] Connected to a master server at 127.152.231.254:43929
I20251027 02:50:38.124924 1736218 heartbeater.cc:461] Registering TS with master...
I20251027 02:50:38.125733 1736218 heartbeater.cc:507] Master 127.152.231.254:43929 requested a full tablet report, sending...
I20251027 02:50:38.128912 1735247 ts_manager.cc:194] Registered new tserver with Master: 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035)
W20251027 02:50:38.715088 1736215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251027 02:50:38.715956 1736215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251027 02:50:38.717092 1736215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251027 02:50:38.819574 1736215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251027 02:50:38.821107 1736215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.152.231.195
I20251027 02:50:38.921195 1736215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.152.231.212:33957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.152.231.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-2/data/info.pb
--webserver_certificate_file=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/test.cert
--webserver_doc_root=/home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www
--webserver_interface=127.152.231.195
--webserver_port=0
--webserver_private_key_file=<redacted>
--webserver_private_key_password_cmd=<redacted>
--tserver_master_addrs=127.152.231.254:43929,127.152.231.253:46719,127.152.231.252:35507
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.152.231.195
--log_dir=/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 1d0cb013a5aad754f2ed06c25f5ec3d8a1873622
build type FASTDEBUG
built by None at 27 Oct 2025 02:43:21 UTC on 5fd53c4cbb9d
build id 8737
TSAN enabled
I20251027 02:50:38.923348 1736215 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251027 02:50:38.925943 1736215 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251027 02:50:38.988900 1736259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251027 02:50:39.121171 1736213 heartbeater.cc:499] Master 127.152.231.252:35507 was elected leader, sending a full tablet report...
W20251027 02:50:39.005609 1736262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:50:40.361444 1736257 debug-util.cc:398] Leaking SignalData structure 0x7b0800034920 after lost signal to thread 1736215
W20251027 02:50:40.839759 1736215 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.834s user 0.694s sys 0.650s
W20251027 02:50:40.840158 1736215 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.834s user 0.694s sys 0.650s
W20251027 02:50:40.860212 1736265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251027 02:50:40.868418 1736263 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection timed out after 1848 milliseconds
I20251027 02:50:40.868503 1736215 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20251027 02:50:40.870388 1736215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251027 02:50:40.891901 1736215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251027 02:50:40.904687 1736215 hybrid_clock.cc:648] HybridClock initialized: now 1761533440904609 us; error 57 us; skew 500 ppm
I20251027 02:50:40.906118 1736215 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251027 02:50:40.974159 1736215 webserver.cc:492] Webserver started at https://127.152.231.195:34915/ using document root /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/../../../www and password file <none>
I20251027 02:50:40.976056 1736215 fs_manager.cc:362] Metadata directory not provided
I20251027 02:50:40.976481 1736215 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251027 02:50:40.977309 1736215 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251027 02:50:40.986155 1736215 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-2/data/instance:
uuid: "6193343d125c4a5d96c9416e0c07b34c"
format_stamp: "Formatted at 2025-10-27 02:50:40 on 5fd53c4cbb9d"
I20251027 02:50:40.988124 1736215 fs_manager.cc:1068] Generated new instance metadata in path /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-2/wal/instance:
uuid: "6193343d125c4a5d96c9416e0c07b34c"
format_stamp: "Formatted at 2025-10-27 02:50:40 on 5fd53c4cbb9d"
I20251027 02:50:41.019433 1736215 fs_manager.cc:696] Time spent creating directory manager: real 0.030s user 0.017s sys 0.011s
I20251027 02:50:41.028764 1736347 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251027 02:50:41.032330 1736215 fs_manager.cc:730] Time spent opening block manager: real 0.008s user 0.005s sys 0.000s
I20251027 02:50:41.032810 1736215 fs_manager.cc:647] Opened local filesystem: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-2/data,/tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-2/wal
uuid: "6193343d125c4a5d96c9416e0c07b34c"
format_stamp: "Formatted at 2025-10-27 02:50:40 on 5fd53c4cbb9d"
I20251027 02:50:41.033346 1736215 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-2/wal
metadata directory: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-2/wal
1 data directories: /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251027 02:50:41.209374 1736215 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251027 02:50:41.211436 1736215 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251027 02:50:41.212145 1736215 kserver.cc:163] Server-wide thread pool size limit: 96588
I20251027 02:50:41.229346 1736215 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251027 02:50:41.246562 1736215 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251027 02:50:41.246928 1736215 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20251027 02:50:41.247315 1736215 ts_tablet_manager.cc:616] Registered 0 tablets
I20251027 02:50:41.247583 1736215 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251027 02:50:43.330857 1736215 rpc_server.cc:307] RPC server started. Bound to: 127.152.231.195:35287
I20251027 02:50:43.331034 1736499 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.152.231.195:35287 every 8 connection(s)
I20251027 02:50:43.334638 1736215 server_base.cc:1179] Dumped server information to /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-2/data/info.pb
I20251027 02:50:43.337972 1729439 external_mini_cluster.cc:1428] Started /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu as pid 1736215
I20251027 02:50:43.338507 1729439 external_mini_cluster.cc:1442] Reading /tmp/build_and_test_flaky/8737/webserver-crawl-itest.0.WebserverCrawlITest.TestAllWebPages_3.1761533243417696-1729439-0/minicluster-data/ts-2/wal/instance
I20251027 02:50:43.436584 1736507 heartbeater.cc:344] Connected to a master server at 127.152.231.253:46719
I20251027 02:50:43.437217 1736507 heartbeater.cc:461] Registering TS with master...
I20251027 02:50:43.438863 1736507 heartbeater.cc:507] Master 127.152.231.253:46719 requested a full tablet report, sending...
I20251027 02:50:43.442266 1735369 ts_manager.cc:194] Registered new tserver with Master: 6193343d125c4a5d96c9416e0c07b34c (127.152.231.195:35287)
I20251027 02:50:43.448438 1736512 heartbeater.cc:344] Connected to a master server at 127.152.231.254:43929
I20251027 02:50:43.448872 1736512 heartbeater.cc:461] Registering TS with master...
I20251027 02:50:43.449291 1736502 heartbeater.cc:344] Connected to a master server at 127.152.231.252:35507
I20251027 02:50:43.449677 1736502 heartbeater.cc:461] Registering TS with master...
I20251027 02:50:43.450059 1736512 heartbeater.cc:507] Master 127.152.231.254:43929 requested a full tablet report, sending...
I20251027 02:50:43.450659 1736502 heartbeater.cc:507] Master 127.152.231.252:35507 requested a full tablet report, sending...
I20251027 02:50:43.452734 1735247 ts_manager.cc:194] Registered new tserver with Master: 6193343d125c4a5d96c9416e0c07b34c (127.152.231.195:35287)
I20251027 02:50:43.453384 1735559 ts_manager.cc:194] Registered new tserver with Master: 6193343d125c4a5d96c9416e0c07b34c (127.152.231.195:35287)
I20251027 02:50:43.455381 1735559 master_service.cc:502] Signed X509 certificate for tserver {username='jenkins-slave'} at 127.152.231.195:60697
I20251027 02:50:43.473416 1729439 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251027 02:50:43.487725 1729439 test_util.cc:276] Using random seed: -148326449
I20251027 02:50:43.621534 1735559 catalog_manager.cc:2257] Servicing CreateTable request from {username='jenkins-slave'} at 127.0.0.1:39866:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\004\001\000UUU\025\004\001\000\252\252\252*\004\001\000\377\377\377?\004\001\000TUUU\004\001\000\251\252\252j"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251027 02:50:43.627113 1735559 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251027 02:50:43.849650 1736424 tablet_service.cc:1505] Processing CreateTablet for tablet 99ba20e0eb6f4f68af16d2c2ce185e0d (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20251027 02:50:43.852732 1736424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99ba20e0eb6f4f68af16d2c2ce185e0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.857995 1736422 tablet_service.cc:1505] Processing CreateTablet for tablet 1bcf8e0642b24b4085dc6ace7e505bfb (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20251027 02:50:43.857955 1736421 tablet_service.cc:1505] Processing CreateTablet for tablet 6f600007e89e4d68bc1e555bdb09f349 (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20251027 02:50:43.858507 1736423 tablet_service.cc:1505] Processing CreateTablet for tablet f233e93dbcdb4a9294aa6c6b9eeb2c40 (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20251027 02:50:43.860129 1736422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bcf8e0642b24b4085dc6ace7e505bfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.857796 1736420 tablet_service.cc:1505] Processing CreateTablet for tablet 13f40d75e47c4380b221402efef87ee4 (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20251027 02:50:43.859109 1736425 tablet_service.cc:1505] Processing CreateTablet for tablet af2834bb9e274a47a69d2c32eb2a9245 (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION VALUES < 357913941
I20251027 02:50:43.861014 1736421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f600007e89e4d68bc1e555bdb09f349. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.864116 1736425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af2834bb9e274a47a69d2c32eb2a9245. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.865942 1736423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f233e93dbcdb4a9294aa6c6b9eeb2c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.867728 1736420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13f40d75e47c4380b221402efef87ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.924516 1735800 tablet_service.cc:1505] Processing CreateTablet for tablet 13f40d75e47c4380b221402efef87ee4 (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20251027 02:50:43.924574 1735804 tablet_service.cc:1505] Processing CreateTablet for tablet 99ba20e0eb6f4f68af16d2c2ce185e0d (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20251027 02:50:43.923395 1735805 tablet_service.cc:1505] Processing CreateTablet for tablet af2834bb9e274a47a69d2c32eb2a9245 (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION VALUES < 357913941
I20251027 02:50:43.924592 1735803 tablet_service.cc:1505] Processing CreateTablet for tablet f233e93dbcdb4a9294aa6c6b9eeb2c40 (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20251027 02:50:43.925523 1735801 tablet_service.cc:1505] Processing CreateTablet for tablet 6f600007e89e4d68bc1e555bdb09f349 (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20251027 02:50:43.923399 1735802 tablet_service.cc:1505] Processing CreateTablet for tablet 1bcf8e0642b24b4085dc6ace7e505bfb (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20251027 02:50:43.929953 1735800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13f40d75e47c4380b221402efef87ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.930907 1735803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f233e93dbcdb4a9294aa6c6b9eeb2c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.932065 1735801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f600007e89e4d68bc1e555bdb09f349. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.933843 1735805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af2834bb9e274a47a69d2c32eb2a9245. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.934579 1735802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bcf8e0642b24b4085dc6ace7e505bfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.933526 1736143 tablet_service.cc:1505] Processing CreateTablet for tablet f233e93dbcdb4a9294aa6c6b9eeb2c40 (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20251027 02:50:43.936743 1736143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f233e93dbcdb4a9294aa6c6b9eeb2c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.941447 1735804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99ba20e0eb6f4f68af16d2c2ce185e0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.941394 1736142 tablet_service.cc:1505] Processing CreateTablet for tablet 1bcf8e0642b24b4085dc6ace7e505bfb (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20251027 02:50:43.938971 1736144 tablet_service.cc:1505] Processing CreateTablet for tablet 99ba20e0eb6f4f68af16d2c2ce185e0d (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20251027 02:50:43.946579 1736142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bcf8e0642b24b4085dc6ace7e505bfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.937847 1736140 tablet_service.cc:1505] Processing CreateTablet for tablet 6f600007e89e4d68bc1e555bdb09f349 (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20251027 02:50:43.948057 1736144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99ba20e0eb6f4f68af16d2c2ce185e0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.940527 1736138 tablet_service.cc:1505] Processing CreateTablet for tablet 13f40d75e47c4380b221402efef87ee4 (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20251027 02:50:43.947392 1736145 tablet_service.cc:1505] Processing CreateTablet for tablet af2834bb9e274a47a69d2c32eb2a9245 (DEFAULT_TABLE table=test-workload [id=43231b1737d0464cafbecad8c62ddcb4]), partition=RANGE (key) PARTITION VALUES < 357913941
I20251027 02:50:43.950205 1736140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f600007e89e4d68bc1e555bdb09f349. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.955888 1736570 tablet_bootstrap.cc:492] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c: Bootstrap starting.
I20251027 02:50:43.964319 1736570 tablet_bootstrap.cc:654] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:43.966288 1736138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13f40d75e47c4380b221402efef87ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.966521 1736570 log.cc:826] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c: Log is configured to *not* fsync() on all Append() calls
I20251027 02:50:43.967007 1736145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af2834bb9e274a47a69d2c32eb2a9245. 1 dirs total, 0 dirs full, 0 dirs failed
I20251027 02:50:43.973570 1736570 tablet_bootstrap.cc:492] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c: No bootstrap required, opened a new log
I20251027 02:50:43.974253 1736570 ts_tablet_manager.cc:1403] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.011s
I20251027 02:50:44.001099 1736570 raft_consensus.cc:359] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.002250 1736570 raft_consensus.cc:385] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.002614 1736570 raft_consensus.cc:740] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6193343d125c4a5d96c9416e0c07b34c, State: Initialized, Role: FOLLOWER
I20251027 02:50:44.003728 1736570 consensus_queue.cc:260] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.026393 1736502 heartbeater.cc:499] Master 127.152.231.252:35507 was elected leader, sending a full tablet report...
I20251027 02:50:44.027148 1736570 ts_tablet_manager.cc:1434] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c: Time spent starting tablet: real 0.053s user 0.028s sys 0.024s
I20251027 02:50:44.030504 1736573 tablet_bootstrap.cc:492] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9: Bootstrap starting.
I20251027 02:50:44.031303 1736570 tablet_bootstrap.cc:492] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c: Bootstrap starting.
I20251027 02:50:44.041429 1736573 tablet_bootstrap.cc:654] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:44.041594 1736570 tablet_bootstrap.cc:654] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:44.045107 1736573 log.cc:826] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9: Log is configured to *not* fsync() on all Append() calls
I20251027 02:50:44.059617 1736574 tablet_bootstrap.cc:492] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40: Bootstrap starting.
I20251027 02:50:44.063545 1736570 tablet_bootstrap.cc:492] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c: No bootstrap required, opened a new log
I20251027 02:50:44.064023 1736570 ts_tablet_manager.cc:1403] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c: Time spent bootstrapping tablet: real 0.033s user 0.019s sys 0.003s
I20251027 02:50:44.067281 1736570 raft_consensus.cc:359] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.067996 1736570 raft_consensus.cc:385] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.068336 1736570 raft_consensus.cc:740] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6193343d125c4a5d96c9416e0c07b34c, State: Initialized, Role: FOLLOWER
I20251027 02:50:44.068995 1736570 consensus_queue.cc:260] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.070767 1736574 tablet_bootstrap.cc:654] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:44.070989 1736570 ts_tablet_manager.cc:1434] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20251027 02:50:44.071801 1736570 tablet_bootstrap.cc:492] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c: Bootstrap starting.
I20251027 02:50:44.076835 1736573 tablet_bootstrap.cc:492] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9: No bootstrap required, opened a new log
I20251027 02:50:44.077438 1736573 ts_tablet_manager.cc:1403] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9: Time spent bootstrapping tablet: real 0.048s user 0.014s sys 0.008s
I20251027 02:50:44.084371 1736570 tablet_bootstrap.cc:654] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:44.087322 1736574 log.cc:826] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40: Log is configured to *not* fsync() on all Append() calls
I20251027 02:50:44.112785 1736570 tablet_bootstrap.cc:492] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c: No bootstrap required, opened a new log
I20251027 02:50:44.113266 1736570 ts_tablet_manager.cc:1403] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c: Time spent bootstrapping tablet: real 0.042s user 0.023s sys 0.008s
I20251027 02:50:44.115564 1736570 raft_consensus.cc:359] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.116226 1736570 raft_consensus.cc:385] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.116499 1736570 raft_consensus.cc:740] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6193343d125c4a5d96c9416e0c07b34c, State: Initialized, Role: FOLLOWER
I20251027 02:50:44.116799 1736574 tablet_bootstrap.cc:492] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40: No bootstrap required, opened a new log
I20251027 02:50:44.117414 1736574 ts_tablet_manager.cc:1403] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40: Time spent bootstrapping tablet: real 0.058s user 0.016s sys 0.009s
I20251027 02:50:44.117127 1736570 consensus_queue.cc:260] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.118863 1736573 raft_consensus.cc:359] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.133263 1736573 raft_consensus.cc:385] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.133636 1736573 raft_consensus.cc:740] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b6ed712e4b46fa8519a5cfde23ccb9, State: Initialized, Role: FOLLOWER
I20251027 02:50:44.134756 1736570 ts_tablet_manager.cc:1434] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c: Time spent starting tablet: real 0.021s user 0.005s sys 0.000s
I20251027 02:50:44.135062 1736573 consensus_queue.cc:260] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.135718 1736570 tablet_bootstrap.cc:492] T 1bcf8e0642b24b4085dc6ace7e505bfb P 6193343d125c4a5d96c9416e0c07b34c: Bootstrap starting.
I20251027 02:50:44.141193 1736570 tablet_bootstrap.cc:654] T 1bcf8e0642b24b4085dc6ace7e505bfb P 6193343d125c4a5d96c9416e0c07b34c: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:44.146384 1736574 raft_consensus.cc:359] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.147581 1736574 raft_consensus.cc:385] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.147936 1736574 raft_consensus.cc:740] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba7dd0e342744505993460bef2180d40, State: Initialized, Role: FOLLOWER
I20251027 02:50:44.149396 1736573 ts_tablet_manager.cc:1434] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9: Time spent starting tablet: real 0.072s user 0.031s sys 0.009s
I20251027 02:50:44.149057 1736574 consensus_queue.cc:260] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.157483 1736573 tablet_bootstrap.cc:492] T 13f40d75e47c4380b221402efef87ee4 P 06b6ed712e4b46fa8519a5cfde23ccb9: Bootstrap starting.
I20251027 02:50:44.158805 1736580 raft_consensus.cc:493] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:50:44.163663 1736572 raft_consensus.cc:493] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:50:44.164324 1736572 raft_consensus.cc:515] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.167409 1736574 ts_tablet_manager.cc:1434] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40: Time spent starting tablet: real 0.050s user 0.035s sys 0.013s
I20251027 02:50:44.160089 1736580 raft_consensus.cc:515] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.171015 1736574 tablet_bootstrap.cc:492] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40: Bootstrap starting.
I20251027 02:50:44.172442 1736573 tablet_bootstrap.cc:654] T 13f40d75e47c4380b221402efef87ee4 P 06b6ed712e4b46fa8519a5cfde23ccb9: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:44.174700 1736580 leader_election.cc:290] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985), 6193343d125c4a5d96c9416e0c07b34c (127.152.231.195:35287)
I20251027 02:50:44.167807 1736572 leader_election.cc:290] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985), 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035)
I20251027 02:50:44.182294 1736574 tablet_bootstrap.cc:654] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:44.184760 1736572 raft_consensus.cc:493] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:50:44.185243 1736572 raft_consensus.cc:515] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.187034 1736572 leader_election.cc:290] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985), 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035)
I20251027 02:50:44.192375 1736573 tablet_bootstrap.cc:492] T 13f40d75e47c4380b221402efef87ee4 P 06b6ed712e4b46fa8519a5cfde23ccb9: No bootstrap required, opened a new log
I20251027 02:50:44.192818 1736573 ts_tablet_manager.cc:1403] T 13f40d75e47c4380b221402efef87ee4 P 06b6ed712e4b46fa8519a5cfde23ccb9: Time spent bootstrapping tablet: real 0.036s user 0.009s sys 0.004s
I20251027 02:50:44.195994 1736573 raft_consensus.cc:359] T 13f40d75e47c4380b221402efef87ee4 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.196902 1736573 raft_consensus.cc:385] T 13f40d75e47c4380b221402efef87ee4 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.197252 1736573 raft_consensus.cc:740] T 13f40d75e47c4380b221402efef87ee4 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b6ed712e4b46fa8519a5cfde23ccb9, State: Initialized, Role: FOLLOWER
I20251027 02:50:44.198074 1736573 consensus_queue.cc:260] T 13f40d75e47c4380b221402efef87ee4 P 06b6ed712e4b46fa8519a5cfde23ccb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.200186 1736573 ts_tablet_manager.cc:1434] T 13f40d75e47c4380b221402efef87ee4 P 06b6ed712e4b46fa8519a5cfde23ccb9: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20251027 02:50:44.200342 1736570 tablet_bootstrap.cc:492] T 1bcf8e0642b24b4085dc6ace7e505bfb P 6193343d125c4a5d96c9416e0c07b34c: No bootstrap required, opened a new log
I20251027 02:50:44.200762 1736570 ts_tablet_manager.cc:1403] T 1bcf8e0642b24b4085dc6ace7e505bfb P 6193343d125c4a5d96c9416e0c07b34c: Time spent bootstrapping tablet: real 0.065s user 0.013s sys 0.005s
I20251027 02:50:44.201040 1736573 tablet_bootstrap.cc:492] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 06b6ed712e4b46fa8519a5cfde23ccb9: Bootstrap starting.
I20251027 02:50:44.202953 1736570 raft_consensus.cc:359] T 1bcf8e0642b24b4085dc6ace7e505bfb P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.203678 1736570 raft_consensus.cc:385] T 1bcf8e0642b24b4085dc6ace7e505bfb P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.203938 1736570 raft_consensus.cc:740] T 1bcf8e0642b24b4085dc6ace7e505bfb P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6193343d125c4a5d96c9416e0c07b34c, State: Initialized, Role: FOLLOWER
I20251027 02:50:44.204564 1736570 consensus_queue.cc:260] T 1bcf8e0642b24b4085dc6ace7e505bfb P 6193343d125c4a5d96c9416e0c07b34c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.207302 1736570 ts_tablet_manager.cc:1434] T 1bcf8e0642b24b4085dc6ace7e505bfb P 6193343d125c4a5d96c9416e0c07b34c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20251027 02:50:44.208187 1736570 tablet_bootstrap.cc:492] T af2834bb9e274a47a69d2c32eb2a9245 P 6193343d125c4a5d96c9416e0c07b34c: Bootstrap starting.
I20251027 02:50:44.212082 1735837 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "99ba20e0eb6f4f68af16d2c2ce185e0d" candidate_uuid: "6193343d125c4a5d96c9416e0c07b34c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba7dd0e342744505993460bef2180d40" is_pre_election: true
I20251027 02:50:44.212287 1735834 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13f40d75e47c4380b221402efef87ee4" candidate_uuid: "6193343d125c4a5d96c9416e0c07b34c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba7dd0e342744505993460bef2180d40" is_pre_election: true
I20251027 02:50:44.214141 1736570 tablet_bootstrap.cc:654] T af2834bb9e274a47a69d2c32eb2a9245 P 6193343d125c4a5d96c9416e0c07b34c: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:44.214277 1736572 raft_consensus.cc:493] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251027 02:50:44.214690 1736352 leader_election.cc:343] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985): Illegal state: must be running to vote when last-logged opid is not known
I20251027 02:50:44.214748 1736572 raft_consensus.cc:515] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
W20251027 02:50:44.215509 1736352 leader_election.cc:343] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985): Illegal state: must be running to vote when last-logged opid is not known
I20251027 02:50:44.216564 1736572 leader_election.cc:290] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985), 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035)
I20251027 02:50:44.217355 1735834 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f600007e89e4d68bc1e555bdb09f349" candidate_uuid: "6193343d125c4a5d96c9416e0c07b34c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba7dd0e342744505993460bef2180d40" is_pre_election: true
W20251027 02:50:44.218465 1736352 leader_election.cc:343] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985): Illegal state: must be running to vote when last-logged opid is not known
W20251027 02:50:44.220599 1736522 tablet.cc:2378] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:50:44.224020 1736573 tablet_bootstrap.cc:654] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 06b6ed712e4b46fa8519a5cfde23ccb9: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:44.225036 1736574 tablet_bootstrap.cc:492] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40: No bootstrap required, opened a new log
I20251027 02:50:44.223825 1736165 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13f40d75e47c4380b221402efef87ee4" candidate_uuid: "6193343d125c4a5d96c9416e0c07b34c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" is_pre_election: true
I20251027 02:50:44.225493 1736574 ts_tablet_manager.cc:1403] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40: Time spent bootstrapping tablet: real 0.055s user 0.011s sys 0.012s
I20251027 02:50:44.225807 1736165 raft_consensus.cc:2468] T 13f40d75e47c4380b221402efef87ee4 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6193343d125c4a5d96c9416e0c07b34c in term 0.
I20251027 02:50:44.222054 1736166 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "99ba20e0eb6f4f68af16d2c2ce185e0d" candidate_uuid: "6193343d125c4a5d96c9416e0c07b34c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" is_pre_election: true
I20251027 02:50:44.227834 1736574 raft_consensus.cc:359] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.227784 1736350 leader_election.cc:304] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06b6ed712e4b46fa8519a5cfde23ccb9, 6193343d125c4a5d96c9416e0c07b34c; no voters: ba7dd0e342744505993460bef2180d40
I20251027 02:50:44.228456 1736574 raft_consensus.cc:385] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.228716 1736574 raft_consensus.cc:740] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba7dd0e342744505993460bef2180d40, State: Initialized, Role: FOLLOWER
I20251027 02:50:44.229491 1736572 raft_consensus.cc:2804] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:50:44.229393 1736574 consensus_queue.cc:260] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.229494 1736446 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f600007e89e4d68bc1e555bdb09f349" candidate_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6193343d125c4a5d96c9416e0c07b34c" is_pre_election: true
I20251027 02:50:44.230019 1736572 raft_consensus.cc:493] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:50:44.230353 1736572 raft_consensus.cc:3060] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:44.229708 1736164 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f600007e89e4d68bc1e555bdb09f349" candidate_uuid: "6193343d125c4a5d96c9416e0c07b34c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" is_pre_election: true
I20251027 02:50:44.230505 1736446 raft_consensus.cc:2468] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06b6ed712e4b46fa8519a5cfde23ccb9 in term 0.
I20251027 02:50:44.230896 1736164 raft_consensus.cc:2468] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6193343d125c4a5d96c9416e0c07b34c in term 0.
I20251027 02:50:44.231236 1736574 ts_tablet_manager.cc:1434] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20251027 02:50:44.232102 1736574 tablet_bootstrap.cc:492] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40: Bootstrap starting.
W20251027 02:50:44.232190 1736350 leader_election.cc:343] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035): Illegal state: must be running to vote when last-logged opid is not known
I20251027 02:50:44.232607 1736350 leader_election.cc:304] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6193343d125c4a5d96c9416e0c07b34c; no voters: 06b6ed712e4b46fa8519a5cfde23ccb9, ba7dd0e342744505993460bef2180d40
I20251027 02:50:44.236246 1736054 leader_election.cc:304] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b6ed712e4b46fa8519a5cfde23ccb9, 6193343d125c4a5d96c9416e0c07b34c; no voters:
I20251027 02:50:44.237350 1736580 raft_consensus.cc:2804] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:50:44.237741 1736580 raft_consensus.cc:493] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:50:44.238138 1736580 raft_consensus.cc:3060] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:44.246625 1735834 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f600007e89e4d68bc1e555bdb09f349" candidate_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba7dd0e342744505993460bef2180d40" is_pre_election: true
I20251027 02:50:44.248286 1736574 tablet_bootstrap.cc:654] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:44.249235 1736572 raft_consensus.cc:515] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.251369 1736572 leader_election.cc:290] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 1 election: Requested vote from peers ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985), 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035)
I20251027 02:50:44.251623 1736570 tablet_bootstrap.cc:492] T af2834bb9e274a47a69d2c32eb2a9245 P 6193343d125c4a5d96c9416e0c07b34c: No bootstrap required, opened a new log
I20251027 02:50:44.252070 1736570 ts_tablet_manager.cc:1403] T af2834bb9e274a47a69d2c32eb2a9245 P 6193343d125c4a5d96c9416e0c07b34c: Time spent bootstrapping tablet: real 0.044s user 0.009s sys 0.000s
I20251027 02:50:44.251896 1736350 leader_election.cc:304] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06b6ed712e4b46fa8519a5cfde23ccb9, 6193343d125c4a5d96c9416e0c07b34c; no voters: ba7dd0e342744505993460bef2180d40
I20251027 02:50:44.252681 1736572 raft_consensus.cc:2804] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:50:44.251830 1736593 raft_consensus.cc:2749] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251027 02:50:44.252689 1735834 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13f40d75e47c4380b221402efef87ee4" candidate_uuid: "6193343d125c4a5d96c9416e0c07b34c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba7dd0e342744505993460bef2180d40"
I20251027 02:50:44.253021 1736572 raft_consensus.cc:493] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:50:44.253377 1736572 raft_consensus.cc:3060] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Advancing to term 1
W20251027 02:50:44.253805 1736352 leader_election.cc:343] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985): Illegal state: must be running to vote when last-logged opid is not known
I20251027 02:50:44.254599 1736570 raft_consensus.cc:359] T af2834bb9e274a47a69d2c32eb2a9245 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } }
I20251027 02:50:44.255286 1736570 raft_consensus.cc:385] T af2834bb9e274a47a69d2c32eb2a9245 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.255570 1736570 raft_consensus.cc:740] T af2834bb9e274a47a69d2c32eb2a9245 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6193343d125c4a5d96c9416e0c07b34c, State: Initialized, Role: FOLLOWER
I20251027 02:50:44.256213 1736570 consensus_queue.cc:260] T af2834bb9e274a47a69d2c32eb2a9245 P 6193343d125c4a5d96c9416e0c07b34c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } }
W20251027 02:50:44.257653 1736055 leader_election.cc:343] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985): Illegal state: must be running to vote when last-logged opid is not known
I20251027 02:50:44.259143 1736580 raft_consensus.cc:515] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.259428 1736573 tablet_bootstrap.cc:492] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 06b6ed712e4b46fa8519a5cfde23ccb9: No bootstrap required, opened a new log
I20251027 02:50:44.259649 1736570 ts_tablet_manager.cc:1434] T af2834bb9e274a47a69d2c32eb2a9245 P 6193343d125c4a5d96c9416e0c07b34c: Time spent starting tablet: real 0.007s user 0.000s sys 0.004s
I20251027 02:50:44.259433 1736164 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "13f40d75e47c4380b221402efef87ee4" candidate_uuid: "6193343d125c4a5d96c9416e0c07b34c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9"
I20251027 02:50:44.259940 1736573 ts_tablet_manager.cc:1403] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 06b6ed712e4b46fa8519a5cfde23ccb9: Time spent bootstrapping tablet: real 0.059s user 0.009s sys 0.008s
I20251027 02:50:44.260109 1736164 raft_consensus.cc:3060] T 13f40d75e47c4380b221402efef87ee4 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:44.260000 1736572 raft_consensus.cc:515] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.260926 1736580 leader_election.cc:290] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [CANDIDATE]: Term 1 election: Requested vote from peers ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985), 6193343d125c4a5d96c9416e0c07b34c (127.152.231.195:35287)
I20251027 02:50:44.261761 1735834 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f600007e89e4d68bc1e555bdb09f349" candidate_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba7dd0e342744505993460bef2180d40"
I20251027 02:50:44.261914 1736446 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f600007e89e4d68bc1e555bdb09f349" candidate_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6193343d125c4a5d96c9416e0c07b34c"
I20251027 02:50:44.262199 1736572 leader_election.cc:290] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 1 election: Requested vote from peers ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985), 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035)
I20251027 02:50:44.262176 1736573 raft_consensus.cc:359] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.262501 1735837 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f600007e89e4d68bc1e555bdb09f349" candidate_uuid: "6193343d125c4a5d96c9416e0c07b34c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba7dd0e342744505993460bef2180d40"
I20251027 02:50:44.262734 1736446 raft_consensus.cc:2393] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 06b6ed712e4b46fa8519a5cfde23ccb9 in current term 1: Already voted for candidate 6193343d125c4a5d96c9416e0c07b34c in this term.
I20251027 02:50:44.262799 1736573 raft_consensus.cc:385] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.263048 1736573 raft_consensus.cc:740] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b6ed712e4b46fa8519a5cfde23ccb9, State: Initialized, Role: FOLLOWER
W20251027 02:50:44.263622 1736352 leader_election.cc:343] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985): Illegal state: must be running to vote when last-logged opid is not known
I20251027 02:50:44.263585 1736573 consensus_queue.cc:260] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 06b6ed712e4b46fa8519a5cfde23ccb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
W20251027 02:50:44.263634 1736055 leader_election.cc:343] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985): Illegal state: must be running to vote when last-logged opid is not known
I20251027 02:50:44.264611 1736054 leader_election.cc:304] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06b6ed712e4b46fa8519a5cfde23ccb9; no voters: 6193343d125c4a5d96c9416e0c07b34c, ba7dd0e342744505993460bef2180d40
I20251027 02:50:44.265465 1736573 ts_tablet_manager.cc:1434] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 06b6ed712e4b46fa8519a5cfde23ccb9: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20251027 02:50:44.265471 1736166 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f600007e89e4d68bc1e555bdb09f349" candidate_uuid: "6193343d125c4a5d96c9416e0c07b34c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9"
I20251027 02:50:44.266007 1736580 raft_consensus.cc:2749] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20251027 02:50:44.266469 1736573 tablet_bootstrap.cc:492] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9: Bootstrap starting.
I20251027 02:50:44.267395 1736166 raft_consensus.cc:2393] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6193343d125c4a5d96c9416e0c07b34c in current term 1: Already voted for candidate 06b6ed712e4b46fa8519a5cfde23ccb9 in this term.
I20251027 02:50:44.268433 1736350 leader_election.cc:304] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6193343d125c4a5d96c9416e0c07b34c; no voters: 06b6ed712e4b46fa8519a5cfde23ccb9, ba7dd0e342744505993460bef2180d40
I20251027 02:50:44.269156 1736572 raft_consensus.cc:2749] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20251027 02:50:44.260435 1736570 tablet_bootstrap.cc:492] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 6193343d125c4a5d96c9416e0c07b34c: Bootstrap starting.
I20251027 02:50:44.279142 1736570 tablet_bootstrap.cc:654] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 6193343d125c4a5d96c9416e0c07b34c: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:44.283010 1736164 raft_consensus.cc:2468] T 13f40d75e47c4380b221402efef87ee4 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6193343d125c4a5d96c9416e0c07b34c in term 1.
I20251027 02:50:44.283334 1736574 tablet_bootstrap.cc:492] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40: No bootstrap required, opened a new log
I20251027 02:50:44.283782 1736574 ts_tablet_manager.cc:1403] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40: Time spent bootstrapping tablet: real 0.052s user 0.012s sys 0.006s
I20251027 02:50:44.284122 1736350 leader_election.cc:304] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06b6ed712e4b46fa8519a5cfde23ccb9, 6193343d125c4a5d96c9416e0c07b34c; no voters: ba7dd0e342744505993460bef2180d40
I20251027 02:50:44.284427 1736573 tablet_bootstrap.cc:654] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:44.284904 1736572 raft_consensus.cc:2804] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:50:44.286079 1736574 raft_consensus.cc:359] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.286700 1736574 raft_consensus.cc:385] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.286957 1736574 raft_consensus.cc:740] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba7dd0e342744505993460bef2180d40, State: Initialized, Role: FOLLOWER
I20251027 02:50:44.287494 1736572 raft_consensus.cc:697] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [term 1 LEADER]: Becoming Leader. State: Replica: 6193343d125c4a5d96c9416e0c07b34c, State: Running, Role: LEADER
I20251027 02:50:44.287765 1736574 consensus_queue.cc:260] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.288435 1736572 consensus_queue.cc:237] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.289757 1736574 ts_tablet_manager.cc:1434] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20251027 02:50:44.290697 1736574 tablet_bootstrap.cc:492] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40: Bootstrap starting.
I20251027 02:50:44.293289 1736581 raft_consensus.cc:493] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:50:44.305263 1736581 raft_consensus.cc:515] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.308543 1736574 tablet_bootstrap.cc:654] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:44.310122 1735559 catalog_manager.cc:5649] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6193343d125c4a5d96c9416e0c07b34c (127.152.231.195). New cstate: current_term: 1 leader_uuid: "6193343d125c4a5d96c9416e0c07b34c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } health_report { overall_health: HEALTHY } } }
I20251027 02:50:44.311647 1736570 tablet_bootstrap.cc:492] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 6193343d125c4a5d96c9416e0c07b34c: No bootstrap required, opened a new log
I20251027 02:50:44.312095 1736570 ts_tablet_manager.cc:1403] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 6193343d125c4a5d96c9416e0c07b34c: Time spent bootstrapping tablet: real 0.052s user 0.014s sys 0.009s
I20251027 02:50:44.314289 1736570 raft_consensus.cc:359] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.312801 1736581 leader_election.cc:290] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035), 6193343d125c4a5d96c9416e0c07b34c (127.152.231.195:35287)
I20251027 02:50:44.314996 1736570 raft_consensus.cc:385] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.315095 1736573 tablet_bootstrap.cc:492] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9: No bootstrap required, opened a new log
I20251027 02:50:44.315310 1736570 raft_consensus.cc:740] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6193343d125c4a5d96c9416e0c07b34c, State: Initialized, Role: FOLLOWER
I20251027 02:50:44.315577 1736573 ts_tablet_manager.cc:1403] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9: Time spent bootstrapping tablet: real 0.049s user 0.004s sys 0.009s
I20251027 02:50:44.316365 1736570 consensus_queue.cc:260] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 6193343d125c4a5d96c9416e0c07b34c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.318398 1736570 ts_tablet_manager.cc:1434] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 6193343d125c4a5d96c9416e0c07b34c: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20251027 02:50:44.318305 1736573 raft_consensus.cc:359] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.319124 1736573 raft_consensus.cc:385] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.319442 1736573 raft_consensus.cc:740] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b6ed712e4b46fa8519a5cfde23ccb9, State: Initialized, Role: FOLLOWER
I20251027 02:50:44.320187 1736573 consensus_queue.cc:260] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.322491 1736573 ts_tablet_manager.cc:1434] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20251027 02:50:44.323313 1736573 tablet_bootstrap.cc:492] T af2834bb9e274a47a69d2c32eb2a9245 P 06b6ed712e4b46fa8519a5cfde23ccb9: Bootstrap starting.
I20251027 02:50:44.329144 1736573 tablet_bootstrap.cc:654] T af2834bb9e274a47a69d2c32eb2a9245 P 06b6ed712e4b46fa8519a5cfde23ccb9: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:44.336058 1736574 tablet_bootstrap.cc:492] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40: No bootstrap required, opened a new log
I20251027 02:50:44.336591 1736574 ts_tablet_manager.cc:1403] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40: Time spent bootstrapping tablet: real 0.046s user 0.005s sys 0.017s
I20251027 02:50:44.339033 1736574 raft_consensus.cc:359] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } }
I20251027 02:50:44.339727 1736574 raft_consensus.cc:385] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.340040 1736574 raft_consensus.cc:740] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba7dd0e342744505993460bef2180d40, State: Initialized, Role: FOLLOWER
I20251027 02:50:44.340704 1736574 consensus_queue.cc:260] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } }
I20251027 02:50:44.346233 1736446 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "99ba20e0eb6f4f68af16d2c2ce185e0d" candidate_uuid: "ba7dd0e342744505993460bef2180d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6193343d125c4a5d96c9416e0c07b34c" is_pre_election: true
I20251027 02:50:44.346913 1736446 raft_consensus.cc:2468] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba7dd0e342744505993460bef2180d40 in term 0.
I20251027 02:50:44.352798 1736164 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "99ba20e0eb6f4f68af16d2c2ce185e0d" candidate_uuid: "ba7dd0e342744505993460bef2180d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" is_pre_election: true
W20251027 02:50:44.353416 1736221 tablet.cc:2378] T 13f40d75e47c4380b221402efef87ee4 P 06b6ed712e4b46fa8519a5cfde23ccb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:50:44.349390 1735741 leader_election.cc:304] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6193343d125c4a5d96c9416e0c07b34c, ba7dd0e342744505993460bef2180d40; no voters:
W20251027 02:50:44.357534 1735739 leader_election.cc:343] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035): Illegal state: must be running to vote when last-logged opid is not known
I20251027 02:50:44.357874 1736581 raft_consensus.cc:2804] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:50:44.358362 1736581 raft_consensus.cc:493] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:50:44.358815 1736581 raft_consensus.cc:3060] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:44.367643 1736573 tablet_bootstrap.cc:492] T af2834bb9e274a47a69d2c32eb2a9245 P 06b6ed712e4b46fa8519a5cfde23ccb9: No bootstrap required, opened a new log
I20251027 02:50:44.368086 1736573 ts_tablet_manager.cc:1403] T af2834bb9e274a47a69d2c32eb2a9245 P 06b6ed712e4b46fa8519a5cfde23ccb9: Time spent bootstrapping tablet: real 0.045s user 0.003s sys 0.012s
I20251027 02:50:44.371462 1736573 raft_consensus.cc:359] T af2834bb9e274a47a69d2c32eb2a9245 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } }
I20251027 02:50:44.372223 1736573 raft_consensus.cc:385] T af2834bb9e274a47a69d2c32eb2a9245 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.372510 1736573 raft_consensus.cc:740] T af2834bb9e274a47a69d2c32eb2a9245 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b6ed712e4b46fa8519a5cfde23ccb9, State: Initialized, Role: FOLLOWER
I20251027 02:50:44.373227 1736573 consensus_queue.cc:260] T af2834bb9e274a47a69d2c32eb2a9245 P 06b6ed712e4b46fa8519a5cfde23ccb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } }
I20251027 02:50:44.375659 1736573 ts_tablet_manager.cc:1434] T af2834bb9e274a47a69d2c32eb2a9245 P 06b6ed712e4b46fa8519a5cfde23ccb9: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20251027 02:50:44.376650 1736573 tablet_bootstrap.cc:492] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 06b6ed712e4b46fa8519a5cfde23ccb9: Bootstrap starting.
I20251027 02:50:44.378471 1736574 ts_tablet_manager.cc:1434] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40: Time spent starting tablet: real 0.042s user 0.005s sys 0.000s
I20251027 02:50:44.382849 1736581 raft_consensus.cc:515] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.384657 1736581 leader_election.cc:290] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [CANDIDATE]: Term 1 election: Requested vote from peers 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035), 6193343d125c4a5d96c9416e0c07b34c (127.152.231.195:35287)
I20251027 02:50:44.385349 1736574 tablet_bootstrap.cc:492] T 13f40d75e47c4380b221402efef87ee4 P ba7dd0e342744505993460bef2180d40: Bootstrap starting.
I20251027 02:50:44.385352 1736164 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "99ba20e0eb6f4f68af16d2c2ce185e0d" candidate_uuid: "ba7dd0e342744505993460bef2180d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9"
W20251027 02:50:44.386591 1735739 leader_election.cc:343] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035): Illegal state: must be running to vote when last-logged opid is not known
I20251027 02:50:44.386731 1736446 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "99ba20e0eb6f4f68af16d2c2ce185e0d" candidate_uuid: "ba7dd0e342744505993460bef2180d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6193343d125c4a5d96c9416e0c07b34c"
I20251027 02:50:44.386963 1736573 tablet_bootstrap.cc:654] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 06b6ed712e4b46fa8519a5cfde23ccb9: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:44.387272 1736446 raft_consensus.cc:3060] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Advancing to term 1
W20251027 02:50:44.390339 1735931 tablet.cc:2378] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251027 02:50:44.391073 1736574 tablet_bootstrap.cc:654] T 13f40d75e47c4380b221402efef87ee4 P ba7dd0e342744505993460bef2180d40: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:44.407195 1736573 tablet_bootstrap.cc:492] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 06b6ed712e4b46fa8519a5cfde23ccb9: No bootstrap required, opened a new log
I20251027 02:50:44.407631 1736573 ts_tablet_manager.cc:1403] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 06b6ed712e4b46fa8519a5cfde23ccb9: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.008s
I20251027 02:50:44.407526 1736446 raft_consensus.cc:2468] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 6193343d125c4a5d96c9416e0c07b34c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba7dd0e342744505993460bef2180d40 in term 1.
I20251027 02:50:44.408715 1735741 leader_election.cc:304] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6193343d125c4a5d96c9416e0c07b34c, ba7dd0e342744505993460bef2180d40; no voters: 06b6ed712e4b46fa8519a5cfde23ccb9
I20251027 02:50:44.409875 1736581 raft_consensus.cc:2804] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:50:44.409826 1736573 raft_consensus.cc:359] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.410329 1736581 raft_consensus.cc:697] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [term 1 LEADER]: Becoming Leader. State: Replica: ba7dd0e342744505993460bef2180d40, State: Running, Role: LEADER
I20251027 02:50:44.410468 1736573 raft_consensus.cc:385] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.410694 1736573 raft_consensus.cc:740] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06b6ed712e4b46fa8519a5cfde23ccb9, State: Initialized, Role: FOLLOWER
I20251027 02:50:44.411258 1736573 consensus_queue.cc:260] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 06b6ed712e4b46fa8519a5cfde23ccb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.411254 1736581 consensus_queue.cc:237] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.412408 1736574 tablet_bootstrap.cc:492] T 13f40d75e47c4380b221402efef87ee4 P ba7dd0e342744505993460bef2180d40: No bootstrap required, opened a new log
I20251027 02:50:44.412811 1736574 ts_tablet_manager.cc:1403] T 13f40d75e47c4380b221402efef87ee4 P ba7dd0e342744505993460bef2180d40: Time spent bootstrapping tablet: real 0.034s user 0.007s sys 0.004s
I20251027 02:50:44.413221 1736573 ts_tablet_manager.cc:1434] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 06b6ed712e4b46fa8519a5cfde23ccb9: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20251027 02:50:44.415176 1736574 raft_consensus.cc:359] T 13f40d75e47c4380b221402efef87ee4 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.415875 1736574 raft_consensus.cc:385] T 13f40d75e47c4380b221402efef87ee4 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.416196 1736574 raft_consensus.cc:740] T 13f40d75e47c4380b221402efef87ee4 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba7dd0e342744505993460bef2180d40, State: Initialized, Role: FOLLOWER
I20251027 02:50:44.416939 1736574 consensus_queue.cc:260] T 13f40d75e47c4380b221402efef87ee4 P ba7dd0e342744505993460bef2180d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.418968 1736574 ts_tablet_manager.cc:1434] T 13f40d75e47c4380b221402efef87ee4 P ba7dd0e342744505993460bef2180d40: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20251027 02:50:44.419909 1736574 tablet_bootstrap.cc:492] T 1bcf8e0642b24b4085dc6ace7e505bfb P ba7dd0e342744505993460bef2180d40: Bootstrap starting.
I20251027 02:50:44.425488 1736574 tablet_bootstrap.cc:654] T 1bcf8e0642b24b4085dc6ace7e505bfb P ba7dd0e342744505993460bef2180d40: Neither blocks nor log segments found. Creating new log.
I20251027 02:50:44.439014 1735559 catalog_manager.cc:5649] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba7dd0e342744505993460bef2180d40 (127.152.231.193). New cstate: current_term: 1 leader_uuid: "ba7dd0e342744505993460bef2180d40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } health_report { overall_health: UNKNOWN } } }
I20251027 02:50:44.462284 1736581 raft_consensus.cc:493] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:50:44.462714 1736581 raft_consensus.cc:515] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } }
I20251027 02:50:44.464310 1736581 leader_election.cc:290] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6193343d125c4a5d96c9416e0c07b34c (127.152.231.195:35287), 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035)
I20251027 02:50:44.465481 1736164 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af2834bb9e274a47a69d2c32eb2a9245" candidate_uuid: "ba7dd0e342744505993460bef2180d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" is_pre_election: true
I20251027 02:50:44.465668 1736446 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af2834bb9e274a47a69d2c32eb2a9245" candidate_uuid: "ba7dd0e342744505993460bef2180d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6193343d125c4a5d96c9416e0c07b34c" is_pre_election: true
I20251027 02:50:44.466100 1736164 raft_consensus.cc:2468] T af2834bb9e274a47a69d2c32eb2a9245 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba7dd0e342744505993460bef2180d40 in term 0.
I20251027 02:50:44.466223 1736446 raft_consensus.cc:2468] T af2834bb9e274a47a69d2c32eb2a9245 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba7dd0e342744505993460bef2180d40 in term 0.
I20251027 02:50:44.467022 1735739 leader_election.cc:304] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b6ed712e4b46fa8519a5cfde23ccb9, ba7dd0e342744505993460bef2180d40; no voters:
I20251027 02:50:44.467753 1736581 raft_consensus.cc:2804] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:50:44.468040 1736581 raft_consensus.cc:493] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:50:44.468194 1736574 tablet_bootstrap.cc:492] T 1bcf8e0642b24b4085dc6ace7e505bfb P ba7dd0e342744505993460bef2180d40: No bootstrap required, opened a new log
I20251027 02:50:44.468292 1736581 raft_consensus.cc:3060] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:44.468647 1736574 ts_tablet_manager.cc:1403] T 1bcf8e0642b24b4085dc6ace7e505bfb P ba7dd0e342744505993460bef2180d40: Time spent bootstrapping tablet: real 0.049s user 0.005s sys 0.017s
I20251027 02:50:44.471084 1736574 raft_consensus.cc:359] T 1bcf8e0642b24b4085dc6ace7e505bfb P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.471745 1736574 raft_consensus.cc:385] T 1bcf8e0642b24b4085dc6ace7e505bfb P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251027 02:50:44.472010 1736574 raft_consensus.cc:740] T 1bcf8e0642b24b4085dc6ace7e505bfb P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba7dd0e342744505993460bef2180d40, State: Initialized, Role: FOLLOWER
I20251027 02:50:44.472666 1736574 consensus_queue.cc:260] T 1bcf8e0642b24b4085dc6ace7e505bfb P ba7dd0e342744505993460bef2180d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.473781 1736581 raft_consensus.cc:515] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } }
I20251027 02:50:44.474843 1736574 ts_tablet_manager.cc:1434] T 1bcf8e0642b24b4085dc6ace7e505bfb P ba7dd0e342744505993460bef2180d40: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20251027 02:50:44.476716 1736581 leader_election.cc:290] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [CANDIDATE]: Term 1 election: Requested vote from peers 6193343d125c4a5d96c9416e0c07b34c (127.152.231.195:35287), 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035)
I20251027 02:50:44.477092 1736446 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af2834bb9e274a47a69d2c32eb2a9245" candidate_uuid: "ba7dd0e342744505993460bef2180d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6193343d125c4a5d96c9416e0c07b34c"
I20251027 02:50:44.477597 1736446 raft_consensus.cc:3060] T af2834bb9e274a47a69d2c32eb2a9245 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:44.478899 1736164 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "af2834bb9e274a47a69d2c32eb2a9245" candidate_uuid: "ba7dd0e342744505993460bef2180d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9"
I20251027 02:50:44.479442 1736164 raft_consensus.cc:3060] T af2834bb9e274a47a69d2c32eb2a9245 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:44.483161 1736446 raft_consensus.cc:2468] T af2834bb9e274a47a69d2c32eb2a9245 P 6193343d125c4a5d96c9416e0c07b34c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba7dd0e342744505993460bef2180d40 in term 1.
I20251027 02:50:44.484117 1735741 leader_election.cc:304] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6193343d125c4a5d96c9416e0c07b34c, ba7dd0e342744505993460bef2180d40; no voters:
I20251027 02:50:44.484910 1736581 raft_consensus.cc:2804] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:50:44.485298 1736581 raft_consensus.cc:697] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [term 1 LEADER]: Becoming Leader. State: Replica: ba7dd0e342744505993460bef2180d40, State: Running, Role: LEADER
I20251027 02:50:44.485268 1736164 raft_consensus.cc:2468] T af2834bb9e274a47a69d2c32eb2a9245 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba7dd0e342744505993460bef2180d40 in term 1.
I20251027 02:50:44.485877 1736581 consensus_queue.cc:237] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } }
I20251027 02:50:44.504814 1735559 catalog_manager.cc:5649] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba7dd0e342744505993460bef2180d40 (127.152.231.193). New cstate: current_term: 1 leader_uuid: "ba7dd0e342744505993460bef2180d40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } health_report { overall_health: HEALTHY } } }
I20251027 02:50:44.545519 1736581 raft_consensus.cc:493] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:50:44.545955 1736581 raft_consensus.cc:515] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.547596 1736581 leader_election.cc:290] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035), 6193343d125c4a5d96c9416e0c07b34c (127.152.231.195:35287)
I20251027 02:50:44.548478 1736164 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f233e93dbcdb4a9294aa6c6b9eeb2c40" candidate_uuid: "ba7dd0e342744505993460bef2180d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" is_pre_election: true
I20251027 02:50:44.548983 1736446 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f233e93dbcdb4a9294aa6c6b9eeb2c40" candidate_uuid: "ba7dd0e342744505993460bef2180d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6193343d125c4a5d96c9416e0c07b34c" is_pre_election: true
I20251027 02:50:44.549131 1736164 raft_consensus.cc:2468] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba7dd0e342744505993460bef2180d40 in term 0.
I20251027 02:50:44.549566 1736446 raft_consensus.cc:2468] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba7dd0e342744505993460bef2180d40 in term 0.
I20251027 02:50:44.550189 1736580 raft_consensus.cc:493] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:50:44.550088 1735739 leader_election.cc:304] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b6ed712e4b46fa8519a5cfde23ccb9, ba7dd0e342744505993460bef2180d40; no voters:
I20251027 02:50:44.550789 1736581 raft_consensus.cc:2804] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:50:44.550585 1736580 raft_consensus.cc:515] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.551095 1736581 raft_consensus.cc:493] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:50:44.551347 1736581 raft_consensus.cc:3060] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:44.552301 1736580 leader_election.cc:290] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985), 6193343d125c4a5d96c9416e0c07b34c (127.152.231.195:35287)
I20251027 02:50:44.553220 1736446 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1bcf8e0642b24b4085dc6ace7e505bfb" candidate_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6193343d125c4a5d96c9416e0c07b34c" is_pre_election: true
I20251027 02:50:44.553169 1735837 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1bcf8e0642b24b4085dc6ace7e505bfb" candidate_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba7dd0e342744505993460bef2180d40" is_pre_election: true
I20251027 02:50:44.553843 1735837 raft_consensus.cc:2468] T 1bcf8e0642b24b4085dc6ace7e505bfb P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06b6ed712e4b46fa8519a5cfde23ccb9 in term 0.
I20251027 02:50:44.553750 1736446 raft_consensus.cc:2468] T 1bcf8e0642b24b4085dc6ace7e505bfb P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06b6ed712e4b46fa8519a5cfde23ccb9 in term 0.
I20251027 02:50:44.554858 1736055 leader_election.cc:304] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b6ed712e4b46fa8519a5cfde23ccb9, ba7dd0e342744505993460bef2180d40; no voters:
I20251027 02:50:44.555640 1736580 raft_consensus.cc:2804] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251027 02:50:44.555958 1736580 raft_consensus.cc:493] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:50:44.556248 1736580 raft_consensus.cc:3060] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:44.556867 1736581 raft_consensus.cc:515] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.558501 1736581 leader_election.cc:290] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [CANDIDATE]: Term 1 election: Requested vote from peers 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035), 6193343d125c4a5d96c9416e0c07b34c (127.152.231.195:35287)
I20251027 02:50:44.559249 1736164 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f233e93dbcdb4a9294aa6c6b9eeb2c40" candidate_uuid: "ba7dd0e342744505993460bef2180d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9"
I20251027 02:50:44.559617 1736446 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f233e93dbcdb4a9294aa6c6b9eeb2c40" candidate_uuid: "ba7dd0e342744505993460bef2180d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6193343d125c4a5d96c9416e0c07b34c"
I20251027 02:50:44.559788 1736164 raft_consensus.cc:3060] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:44.560137 1736446 raft_consensus.cc:3060] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:44.562000 1736580 raft_consensus.cc:515] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.563722 1736580 leader_election.cc:290] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [CANDIDATE]: Term 1 election: Requested vote from peers ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985), 6193343d125c4a5d96c9416e0c07b34c (127.152.231.195:35287)
I20251027 02:50:44.564628 1735837 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1bcf8e0642b24b4085dc6ace7e505bfb" candidate_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba7dd0e342744505993460bef2180d40"
I20251027 02:50:44.564792 1736444 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1bcf8e0642b24b4085dc6ace7e505bfb" candidate_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6193343d125c4a5d96c9416e0c07b34c"
I20251027 02:50:44.565229 1735837 raft_consensus.cc:3060] T 1bcf8e0642b24b4085dc6ace7e505bfb P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:44.565565 1736444 raft_consensus.cc:3060] T 1bcf8e0642b24b4085dc6ace7e505bfb P 6193343d125c4a5d96c9416e0c07b34c [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:44.565865 1736446 raft_consensus.cc:2468] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 6193343d125c4a5d96c9416e0c07b34c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba7dd0e342744505993460bef2180d40 in term 1.
I20251027 02:50:44.566092 1736164 raft_consensus.cc:2468] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba7dd0e342744505993460bef2180d40 in term 1.
I20251027 02:50:44.566897 1735741 leader_election.cc:304] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6193343d125c4a5d96c9416e0c07b34c, ba7dd0e342744505993460bef2180d40; no voters:
I20251027 02:50:44.567652 1736581 raft_consensus.cc:2804] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:50:44.568053 1736581 raft_consensus.cc:697] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [term 1 LEADER]: Becoming Leader. State: Replica: ba7dd0e342744505993460bef2180d40, State: Running, Role: LEADER
I20251027 02:50:44.568683 1736581 consensus_queue.cc:237] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.572338 1736444 raft_consensus.cc:2468] T 1bcf8e0642b24b4085dc6ace7e505bfb P 6193343d125c4a5d96c9416e0c07b34c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06b6ed712e4b46fa8519a5cfde23ccb9 in term 1.
I20251027 02:50:44.573444 1736054 leader_election.cc:304] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06b6ed712e4b46fa8519a5cfde23ccb9, 6193343d125c4a5d96c9416e0c07b34c; no voters:
I20251027 02:50:44.574216 1736580 raft_consensus.cc:2804] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 1 FOLLOWER]: Leader election won for term 1
I20251027 02:50:44.574679 1735837 raft_consensus.cc:2468] T 1bcf8e0642b24b4085dc6ace7e505bfb P ba7dd0e342744505993460bef2180d40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06b6ed712e4b46fa8519a5cfde23ccb9 in term 1.
I20251027 02:50:44.574733 1736580 raft_consensus.cc:697] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 1 LEADER]: Becoming Leader. State: Replica: 06b6ed712e4b46fa8519a5cfde23ccb9, State: Running, Role: LEADER
I20251027 02:50:44.575628 1736580 consensus_queue.cc:237] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.594285 1735559 catalog_manager.cc:5649] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba7dd0e342744505993460bef2180d40 (127.152.231.193). New cstate: current_term: 1 leader_uuid: "ba7dd0e342744505993460bef2180d40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } health_report { overall_health: UNKNOWN } } }
I20251027 02:50:44.602983 1735554 catalog_manager.cc:5649] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194). New cstate: current_term: 1 leader_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } health_report { overall_health: UNKNOWN } } }
I20251027 02:50:44.702184 1736572 consensus_queue.cc:1048] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [LEADER]: Connected to new peer: Peer: permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:50:44.713754 1736581 raft_consensus.cc:493] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:50:44.714300 1736581 raft_consensus.cc:515] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:44.716324 1736581 leader_election.cc:290] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035), 6193343d125c4a5d96c9416e0c07b34c (127.152.231.195:35287)
I20251027 02:50:44.717389 1736166 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f600007e89e4d68bc1e555bdb09f349" candidate_uuid: "ba7dd0e342744505993460bef2180d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" is_pre_election: true
I20251027 02:50:44.717566 1736444 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f600007e89e4d68bc1e555bdb09f349" candidate_uuid: "ba7dd0e342744505993460bef2180d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6193343d125c4a5d96c9416e0c07b34c" is_pre_election: true
I20251027 02:50:44.718464 1736166 raft_consensus.cc:2393] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba7dd0e342744505993460bef2180d40 in current term 1: Already voted for candidate 06b6ed712e4b46fa8519a5cfde23ccb9 in this term.
I20251027 02:50:44.718560 1736444 raft_consensus.cc:2393] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba7dd0e342744505993460bef2180d40 in current term 1: Already voted for candidate 6193343d125c4a5d96c9416e0c07b34c in this term.
I20251027 02:50:44.719887 1735739 leader_election.cc:304] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba7dd0e342744505993460bef2180d40; no voters: 06b6ed712e4b46fa8519a5cfde23ccb9, 6193343d125c4a5d96c9416e0c07b34c
I20251027 02:50:44.720737 1736581 raft_consensus.cc:3060] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:44.727212 1736581 raft_consensus.cc:2749] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251027 02:50:44.752007 1735837 raft_consensus.cc:3060] T 13f40d75e47c4380b221402efef87ee4 P ba7dd0e342744505993460bef2180d40 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:44.760115 1736572 consensus_queue.cc:1048] T 13f40d75e47c4380b221402efef87ee4 P 6193343d125c4a5d96c9416e0c07b34c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:50:44.806705 1736581 consensus_queue.cc:1048] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:50:44.845355 1736164 raft_consensus.cc:3060] T 99ba20e0eb6f4f68af16d2c2ce185e0d P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 0 FOLLOWER]: Advancing to term 1
I20251027 02:50:44.852631 1736581 consensus_queue.cc:1048] T 99ba20e0eb6f4f68af16d2c2ce185e0d P ba7dd0e342744505993460bef2180d40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:50:44.882629 1736581 consensus_queue.cc:1048] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:50:44.906335 1736627 consensus_queue.cc:1048] T af2834bb9e274a47a69d2c32eb2a9245 P ba7dd0e342744505993460bef2180d40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251027 02:50:44.966001 1736580 consensus_queue.cc:1048] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:50:44.969388 1736627 consensus_queue.cc:1048] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:50:44.993773 1736580 consensus_queue.cc:1048] T 1bcf8e0642b24b4085dc6ace7e505bfb P 06b6ed712e4b46fa8519a5cfde23ccb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:50:44.998246 1736609 consensus_queue.cc:1048] T f233e93dbcdb4a9294aa6c6b9eeb2c40 P ba7dd0e342744505993460bef2180d40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:50:45.837901 1736640 raft_consensus.cc:493] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251027 02:50:45.838435 1736640 raft_consensus.cc:515] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:45.840996 1736640 leader_election.cc:290] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985), 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035)
I20251027 02:50:45.841454 1735837 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f600007e89e4d68bc1e555bdb09f349" candidate_uuid: "6193343d125c4a5d96c9416e0c07b34c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba7dd0e342744505993460bef2180d40" is_pre_election: true
I20251027 02:50:45.842113 1735837 raft_consensus.cc:2468] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6193343d125c4a5d96c9416e0c07b34c in term 1.
I20251027 02:50:45.843443 1736352 leader_election.cc:304] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6193343d125c4a5d96c9416e0c07b34c, ba7dd0e342744505993460bef2180d40; no voters:
I20251027 02:50:45.844905 1736640 raft_consensus.cc:2804] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251027 02:50:45.845499 1736640 raft_consensus.cc:493] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251027 02:50:45.847774 1736640 raft_consensus.cc:3060] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 1 FOLLOWER]: Advancing to term 2
I20251027 02:50:45.849054 1736164 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f600007e89e4d68bc1e555bdb09f349" candidate_uuid: "6193343d125c4a5d96c9416e0c07b34c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" is_pre_election: true
I20251027 02:50:45.849658 1736164 raft_consensus.cc:2468] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6193343d125c4a5d96c9416e0c07b34c in term 1.
I20251027 02:50:45.854876 1736640 raft_consensus.cc:515] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:45.858103 1735837 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f600007e89e4d68bc1e555bdb09f349" candidate_uuid: "6193343d125c4a5d96c9416e0c07b34c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba7dd0e342744505993460bef2180d40"
I20251027 02:50:45.858767 1735837 raft_consensus.cc:3060] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40 [term 1 FOLLOWER]: Advancing to term 2
I20251027 02:50:45.858873 1736164 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6f600007e89e4d68bc1e555bdb09f349" candidate_uuid: "6193343d125c4a5d96c9416e0c07b34c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9"
I20251027 02:50:45.859464 1736164 raft_consensus.cc:3060] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 1 FOLLOWER]: Advancing to term 2
I20251027 02:50:45.865314 1735837 raft_consensus.cc:2468] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6193343d125c4a5d96c9416e0c07b34c in term 2.
I20251027 02:50:45.866394 1736352 leader_election.cc:304] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6193343d125c4a5d96c9416e0c07b34c, ba7dd0e342744505993460bef2180d40; no voters:
I20251027 02:50:45.867218 1736644 raft_consensus.cc:2804] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 2 FOLLOWER]: Leader election won for term 2
I20251027 02:50:45.867699 1736644 raft_consensus.cc:697] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [term 2 LEADER]: Becoming Leader. State: Replica: 6193343d125c4a5d96c9416e0c07b34c, State: Running, Role: LEADER
I20251027 02:50:45.867813 1736164 raft_consensus.cc:2468] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6193343d125c4a5d96c9416e0c07b34c in term 2.
I20251027 02:50:45.868474 1736644 consensus_queue.cc:237] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } }
I20251027 02:50:45.869290 1736640 leader_election.cc:290] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [CANDIDATE]: Term 2 election: Requested vote from peers ba7dd0e342744505993460bef2180d40 (127.152.231.193:40985), 06b6ed712e4b46fa8519a5cfde23ccb9 (127.152.231.194:38035)
I20251027 02:50:45.899494 1735554 catalog_manager.cc:5649] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c reported cstate change: term changed from 0 to 2, leader changed from <none> to 6193343d125c4a5d96c9416e0c07b34c (127.152.231.195). New cstate: current_term: 2 leader_uuid: "6193343d125c4a5d96c9416e0c07b34c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6193343d125c4a5d96c9416e0c07b34c" member_type: VOTER last_known_addr { host: "127.152.231.195" port: 35287 } health_report { overall_health: HEALTHY } } }
I20251027 02:50:46.336802 1736164 raft_consensus.cc:1275] T 6f600007e89e4d68bc1e555bdb09f349 P 06b6ed712e4b46fa8519a5cfde23ccb9 [term 2 FOLLOWER]: Refusing update from remote peer 6193343d125c4a5d96c9416e0c07b34c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251027 02:50:46.338368 1736644 consensus_queue.cc:1048] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [LEADER]: Connected to new peer: Peer: permanent_uuid: "06b6ed712e4b46fa8519a5cfde23ccb9" member_type: VOTER last_known_addr { host: "127.152.231.194" port: 38035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:50:46.345108 1735834 raft_consensus.cc:1275] T 6f600007e89e4d68bc1e555bdb09f349 P ba7dd0e342744505993460bef2180d40 [term 2 FOLLOWER]: Refusing update from remote peer 6193343d125c4a5d96c9416e0c07b34c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251027 02:50:46.349711 1736640 consensus_queue.cc:1048] T 6f600007e89e4d68bc1e555bdb09f349 P 6193343d125c4a5d96c9416e0c07b34c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba7dd0e342744505993460bef2180d40" member_type: VOTER last_known_addr { host: "127.152.231.193" port: 40985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251027 02:50:46.395382 1736682 mvcc.cc:204] Tried to move back new op lower bound from 7215240996173946880 to 7215240994288676864. Current Snapshot: MvccSnapshot[applied={T|T < 7215240996173946880}]
W20251027 02:50:51.002116 1736757 debug-util.cc:398] Leaking SignalData structure 0x7b080003e7e0 after lost signal to thread 1735152
W20251027 02:50:51.002966 1736757 debug-util.cc:398] Leaking SignalData structure 0x7b080008e680 after lost signal to thread 1735295
W20251027 02:50:57.950376 1735598 debug-util.cc:398] Leaking SignalData structure 0x7b08000aad80 after lost signal to thread 1735447
W20251027 02:50:57.951200 1735598 debug-util.cc:398] Leaking SignalData structure 0x7b0800051220 after lost signal to thread 1735601
W20251027 02:50:59.325110 1737010 debug-util.cc:398] Leaking SignalData structure 0x7b08000b2760 after lost signal to thread 1735302
W20251027 02:50:59.325882 1737010 debug-util.cc:398] Leaking SignalData structure 0x7b08000afee0 after lost signal to thread 1735429
W20251027 02:51:07.461624 1737245 debug-util.cc:398] Leaking SignalData structure 0x7b08000afda0 after lost signal to thread 1735447
W20251027 02:51:07.462373 1737245 debug-util.cc:398] Leaking SignalData structure 0x7b08000b78c0 after lost signal to thread 1735601
W20251027 02:51:17.885389 1737434 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5780 after lost signal to thread 1735693
W20251027 02:51:17.886634 1737434 debug-util.cc:398] Leaking SignalData structure 0x7b08000b1580 after lost signal to thread 1735907
W20251027 02:51:20.946759 1735598 debug-util.cc:398] Leaking SignalData structure 0x7b08000b4d80 after lost signal to thread 1735447
W20251027 02:51:20.947540 1735598 debug-util.cc:398] Leaking SignalData structure 0x7b08000af1c0 after lost signal to thread 1735601
W20251027 02:51:22.611248 1735598 debug-util.cc:398] Leaking SignalData structure 0x7b08000b13e0 after lost signal to thread 1735447
W20251027 02:51:22.611900 1735598 debug-util.cc:398] Leaking SignalData structure 0x7b08000ad620 after lost signal to thread 1735601
W20251027 02:51:25.565675 1735426 debug-util.cc:398] Leaking SignalData structure 0x7b08000ae8a0 after lost signal to thread 1735302
W20251027 02:51:25.566468 1735426 debug-util.cc:398] Leaking SignalData structure 0x7b0800090ee0 after lost signal to thread 1735429
W20251027 02:51:28.284015 1735292 debug-util.cc:398] Leaking SignalData structure 0x7b08000894a0 after lost signal to thread 1735152
W20251027 02:51:28.284790 1735292 debug-util.cc:398] Leaking SignalData structure 0x7b080009cfa0 after lost signal to thread 1735295
W20251027 02:51:29.156111 1737820 debug-util.cc:398] Leaking SignalData structure 0x7b080006a380 after lost signal to thread 1735938
W20251027 02:51:29.157380 1737820 debug-util.cc:398] Leaking SignalData structure 0x7b080006a760 after lost signal to thread 1736212
W20251027 02:51:40.082274 1738061 debug-util.cc:398] Leaking SignalData structure 0x7b08000eb800 after lost signal to thread 1736234
W20251027 02:51:40.083518 1738061 debug-util.cc:398] Leaking SignalData structure 0x7b08000e94e0 after lost signal to thread 1736499
I20251027 02:51:46.707513 1737434 scanners.cc:360] slow scan reporting is disabled: set --show_slow_scans to enable
I20251027 02:51:47.802543 1737820 scanners.cc:360] slow scan reporting is disabled: set --show_slow_scans to enable
W20251027 02:51:47.926764 1736496 debug-util.cc:398] Leaking SignalData structure 0x7b08000da7c0 after lost signal to thread 1736234
W20251027 02:51:47.927883 1736496 debug-util.cc:398] Leaking SignalData structure 0x7b08000e12e0 after lost signal to thread 1736499
I20251027 02:51:48.802860 1738061 scanners.cc:360] slow scan reporting is disabled: set --show_slow_scans to enable
I20251027 02:51:50.700546 1729439 webserver-crawl-itest.cc:326] Dumping visited URLs
I20251027 02:51:50.700765 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963
I20251027 02:51:50.700896 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/bootstrap/js/bootstrap-table.min.js
I20251027 02:51:50.700984 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/bootstrap/js/bootstrap.min.js
I20251027 02:51:50.701095 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/config
I20251027 02:51:50.701195 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/dashboards
I20251027 02:51:50.701272 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/healthz
I20251027 02:51:50.701351 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/jquery-3.5.1.min.js
I20251027 02:51:50.701431 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/kudu.js
I20251027 02:51:50.701509 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/log-anchors?id=13f40d75e47c4380b221402efef87ee4
I20251027 02:51:50.701586 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/log-anchors?id=1bcf8e0642b24b4085dc6ace7e505bfb
I20251027 02:51:50.701661 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/log-anchors?id=6f600007e89e4d68bc1e555bdb09f349
I20251027 02:51:50.701745 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/log-anchors?id=99ba20e0eb6f4f68af16d2c2ce185e0d
I20251027 02:51:50.701819 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/log-anchors?id=af2834bb9e274a47a69d2c32eb2a9245
I20251027 02:51:50.701897 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/log-anchors?id=f233e93dbcdb4a9294aa6c6b9eeb2c40
I20251027 02:51:50.701972 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/logs
I20251027 02:51:50.702045 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/maintenance-manager
I20251027 02:51:50.702119 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/mem-trackers
I20251027 02:51:50.702191 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/memz
I20251027 02:51:50.702266 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/metrics
I20251027 02:51:50.702340 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/metrics_prometheus
I20251027 02:51:50.702414 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/rpcz
I20251027 02:51:50.702486 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/scans
I20251027 02:51:50.702558 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/stacks
I20251027 02:51:50.702633 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/startup
I20251027 02:51:50.702709 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet-consensus-status?id=13f40d75e47c4380b221402efef87ee4
I20251027 02:51:50.702786 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet-consensus-status?id=1bcf8e0642b24b4085dc6ace7e505bfb
I20251027 02:51:50.702863 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet-consensus-status?id=6f600007e89e4d68bc1e555bdb09f349
I20251027 02:51:50.702947 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet-consensus-status?id=99ba20e0eb6f4f68af16d2c2ce185e0d
I20251027 02:51:50.703024 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet-consensus-status?id=af2834bb9e274a47a69d2c32eb2a9245
I20251027 02:51:50.703099 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet-consensus-status?id=f233e93dbcdb4a9294aa6c6b9eeb2c40
I20251027 02:51:50.703172 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet-rowsetlayout-svg?id=13f40d75e47c4380b221402efef87ee4
I20251027 02:51:50.703246 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet-rowsetlayout-svg?id=1bcf8e0642b24b4085dc6ace7e505bfb
I20251027 02:51:50.703322 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet-rowsetlayout-svg?id=6f600007e89e4d68bc1e555bdb09f349
I20251027 02:51:50.703397 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet-rowsetlayout-svg?id=99ba20e0eb6f4f68af16d2c2ce185e0d
I20251027 02:51:50.703469 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet-rowsetlayout-svg?id=af2834bb9e274a47a69d2c32eb2a9245
I20251027 02:51:50.703542 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet-rowsetlayout-svg?id=f233e93dbcdb4a9294aa6c6b9eeb2c40
I20251027 02:51:50.703616 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet?id=13f40d75e47c4380b221402efef87ee4
I20251027 02:51:50.703694 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet?id=1bcf8e0642b24b4085dc6ace7e505bfb
I20251027 02:51:50.703766 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet?id=6f600007e89e4d68bc1e555bdb09f349
I20251027 02:51:50.703840 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet?id=99ba20e0eb6f4f68af16d2c2ce185e0d
I20251027 02:51:50.703919 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet?id=af2834bb9e274a47a69d2c32eb2a9245
I20251027 02:51:50.703994 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablet?id=f233e93dbcdb4a9294aa6c6b9eeb2c40
I20251027 02:51:50.704066 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/tablets
I20251027 02:51:50.704138 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz
I20251027 02:51:50.704212 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=acceptor%20pool
I20251027 02:51:50.704285 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=all
I20251027 02:51:50.704361 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=cache
I20251027 02:51:50.704433 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=cloud%20detector
I20251027 02:51:50.704506 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=heartbeater
I20251027 02:51:50.704583 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=kernel-watchdog
I20251027 02:51:50.704659 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=maintenance
I20251027 02:51:50.704749 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=ntp
I20251027 02:51:50.704825 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=reactor
I20251027 02:51:50.704983 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=scanners
I20251027 02:51:50.705062 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=server
I20251027 02:51:50.705183 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:51:50.705260 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:51:50.705336 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:51:50.705412 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=service%20pool%20kudu.tserver.TabletServerAdminService
I20251027 02:51:50.705492 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=service%20pool%20kudu.tserver.TabletServerService
I20251027 02:51:50.705569 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=tablet-copy
I20251027 02:51:50.705646 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/threadz?group=thread%20pool
I20251027 02:51:50.705718 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/transactions
I20251027 02:51:50.705796 1729439 webserver-crawl-itest.cc:328] https://127.152.231.193:40963/varz
I20251027 02:51:50.705869 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289
I20251027 02:51:50.706086 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/bootstrap/js/bootstrap-table.min.js
I20251027 02:51:50.706207 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/bootstrap/js/bootstrap.min.js
I20251027 02:51:50.706296 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/config
I20251027 02:51:50.706382 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/dashboards
I20251027 02:51:50.706472 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/healthz
I20251027 02:51:50.706554 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/jquery-3.5.1.min.js
I20251027 02:51:50.706637 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/kudu.js
I20251027 02:51:50.706717 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/log-anchors?id=13f40d75e47c4380b221402efef87ee4
I20251027 02:51:50.706795 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/log-anchors?id=1bcf8e0642b24b4085dc6ace7e505bfb
I20251027 02:51:50.706872 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/log-anchors?id=6f600007e89e4d68bc1e555bdb09f349
I20251027 02:51:50.706971 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/log-anchors?id=99ba20e0eb6f4f68af16d2c2ce185e0d
I20251027 02:51:50.707049 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/log-anchors?id=af2834bb9e274a47a69d2c32eb2a9245
I20251027 02:51:50.707126 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/log-anchors?id=f233e93dbcdb4a9294aa6c6b9eeb2c40
I20251027 02:51:50.707202 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/logs
I20251027 02:51:50.707278 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/maintenance-manager
I20251027 02:51:50.707353 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/mem-trackers
I20251027 02:51:50.707428 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/memz
I20251027 02:51:50.707504 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/metrics
I20251027 02:51:50.707580 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/metrics_prometheus
I20251027 02:51:50.707655 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/rpcz
I20251027 02:51:50.707734 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/scans
I20251027 02:51:50.707813 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/stacks
I20251027 02:51:50.707894 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/startup
I20251027 02:51:50.707978 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet-consensus-status?id=13f40d75e47c4380b221402efef87ee4
I20251027 02:51:50.708065 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet-consensus-status?id=1bcf8e0642b24b4085dc6ace7e505bfb
I20251027 02:51:50.708146 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet-consensus-status?id=6f600007e89e4d68bc1e555bdb09f349
I20251027 02:51:50.708228 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet-consensus-status?id=99ba20e0eb6f4f68af16d2c2ce185e0d
I20251027 02:51:50.708308 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet-consensus-status?id=af2834bb9e274a47a69d2c32eb2a9245
I20251027 02:51:50.708387 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet-consensus-status?id=f233e93dbcdb4a9294aa6c6b9eeb2c40
I20251027 02:51:50.708465 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet-rowsetlayout-svg?id=13f40d75e47c4380b221402efef87ee4
I20251027 02:51:50.708544 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet-rowsetlayout-svg?id=1bcf8e0642b24b4085dc6ace7e505bfb
I20251027 02:51:50.708618 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet-rowsetlayout-svg?id=6f600007e89e4d68bc1e555bdb09f349
I20251027 02:51:50.708695 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet-rowsetlayout-svg?id=99ba20e0eb6f4f68af16d2c2ce185e0d
I20251027 02:51:50.708771 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet-rowsetlayout-svg?id=af2834bb9e274a47a69d2c32eb2a9245
I20251027 02:51:50.708847 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet-rowsetlayout-svg?id=f233e93dbcdb4a9294aa6c6b9eeb2c40
I20251027 02:51:50.708930 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet?id=13f40d75e47c4380b221402efef87ee4
I20251027 02:51:50.709007 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet?id=1bcf8e0642b24b4085dc6ace7e505bfb
I20251027 02:51:50.709096 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet?id=6f600007e89e4d68bc1e555bdb09f349
I20251027 02:51:50.709177 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet?id=99ba20e0eb6f4f68af16d2c2ce185e0d
I20251027 02:51:50.709262 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet?id=af2834bb9e274a47a69d2c32eb2a9245
I20251027 02:51:50.709335 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablet?id=f233e93dbcdb4a9294aa6c6b9eeb2c40
I20251027 02:51:50.709408 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/tablets
I20251027 02:51:50.709487 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz
I20251027 02:51:50.709558 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=acceptor%20pool
I20251027 02:51:50.709630 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=all
I20251027 02:51:50.709703 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=cache
I20251027 02:51:50.709775 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=cloud%20detector
I20251027 02:51:50.709847 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=heartbeater
I20251027 02:51:50.709934 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=kernel-watchdog
I20251027 02:51:50.710011 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=maintenance
I20251027 02:51:50.710089 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=ntp
I20251027 02:51:50.710165 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=reactor
I20251027 02:51:50.710238 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=scanners
I20251027 02:51:50.710311 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=server
I20251027 02:51:50.710388 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:51:50.710484 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:51:50.710561 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:51:50.710647 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=service%20pool%20kudu.tserver.TabletServerAdminService
I20251027 02:51:50.710736 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=service%20pool%20kudu.tserver.TabletServerService
I20251027 02:51:50.710811 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=tablet-copy
I20251027 02:51:50.710893 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/threadz?group=thread%20pool
I20251027 02:51:50.710971 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/transactions
I20251027 02:51:50.711050 1729439 webserver-crawl-itest.cc:328] https://127.152.231.194:40289/varz
I20251027 02:51:50.711123 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915
I20251027 02:51:50.711197 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/bootstrap/js/bootstrap-table.min.js
I20251027 02:51:50.711272 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/bootstrap/js/bootstrap.min.js
I20251027 02:51:50.711347 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/config
I20251027 02:51:50.711422 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/dashboards
I20251027 02:51:50.711498 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/healthz
I20251027 02:51:50.711575 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/jquery-3.5.1.min.js
I20251027 02:51:50.711652 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/kudu.js
I20251027 02:51:50.711727 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/log-anchors?id=13f40d75e47c4380b221402efef87ee4
I20251027 02:51:50.711802 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/log-anchors?id=1bcf8e0642b24b4085dc6ace7e505bfb
I20251027 02:51:50.711875 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/log-anchors?id=6f600007e89e4d68bc1e555bdb09f349
I20251027 02:51:50.711958 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/log-anchors?id=99ba20e0eb6f4f68af16d2c2ce185e0d
I20251027 02:51:50.712034 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/log-anchors?id=af2834bb9e274a47a69d2c32eb2a9245
I20251027 02:51:50.712110 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/log-anchors?id=f233e93dbcdb4a9294aa6c6b9eeb2c40
I20251027 02:51:50.712183 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/logs
I20251027 02:51:50.712257 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/maintenance-manager
I20251027 02:51:50.712332 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/mem-trackers
I20251027 02:51:50.712405 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/memz
I20251027 02:51:50.712481 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/metrics
I20251027 02:51:50.712556 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/metrics_prometheus
I20251027 02:51:50.712630 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/rpcz
I20251027 02:51:50.712704 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/scans
I20251027 02:51:50.712778 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/stacks
I20251027 02:51:50.712852 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/startup
I20251027 02:51:50.712934 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet-consensus-status?id=13f40d75e47c4380b221402efef87ee4
I20251027 02:51:50.713014 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet-consensus-status?id=1bcf8e0642b24b4085dc6ace7e505bfb
I20251027 02:51:50.713110 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet-consensus-status?id=6f600007e89e4d68bc1e555bdb09f349
I20251027 02:51:50.713191 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet-consensus-status?id=99ba20e0eb6f4f68af16d2c2ce185e0d
I20251027 02:51:50.713266 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet-consensus-status?id=af2834bb9e274a47a69d2c32eb2a9245
I20251027 02:51:50.713343 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet-consensus-status?id=f233e93dbcdb4a9294aa6c6b9eeb2c40
I20251027 02:51:50.713418 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet-rowsetlayout-svg?id=13f40d75e47c4380b221402efef87ee4
I20251027 02:51:50.713492 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet-rowsetlayout-svg?id=1bcf8e0642b24b4085dc6ace7e505bfb
I20251027 02:51:50.713565 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet-rowsetlayout-svg?id=6f600007e89e4d68bc1e555bdb09f349
I20251027 02:51:50.713639 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet-rowsetlayout-svg?id=99ba20e0eb6f4f68af16d2c2ce185e0d
I20251027 02:51:50.713712 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet-rowsetlayout-svg?id=af2834bb9e274a47a69d2c32eb2a9245
I20251027 02:51:50.713785 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet-rowsetlayout-svg?id=f233e93dbcdb4a9294aa6c6b9eeb2c40
I20251027 02:51:50.713857 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet?id=13f40d75e47c4380b221402efef87ee4
I20251027 02:51:50.713935 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet?id=1bcf8e0642b24b4085dc6ace7e505bfb
I20251027 02:51:50.714010 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet?id=6f600007e89e4d68bc1e555bdb09f349
I20251027 02:51:50.714082 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet?id=99ba20e0eb6f4f68af16d2c2ce185e0d
I20251027 02:51:50.714156 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet?id=af2834bb9e274a47a69d2c32eb2a9245
I20251027 02:51:50.714231 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablet?id=f233e93dbcdb4a9294aa6c6b9eeb2c40
I20251027 02:51:50.714303 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/tablets
I20251027 02:51:50.714377 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz
I20251027 02:51:50.714452 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=acceptor%20pool
I20251027 02:51:50.714525 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=all
I20251027 02:51:50.714600 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=cache
I20251027 02:51:50.714675 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=cloud%20detector
I20251027 02:51:50.714748 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=heartbeater
I20251027 02:51:50.714828 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=kernel-watchdog
I20251027 02:51:50.714910 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=maintenance
I20251027 02:51:50.714991 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=ntp
I20251027 02:51:50.715068 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=reactor
I20251027 02:51:50.715147 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=scanners
I20251027 02:51:50.715221 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=server
I20251027 02:51:50.715301 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:51:50.715376 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:51:50.715453 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:51:50.715530 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=service%20pool%20kudu.tserver.TabletServerAdminService
I20251027 02:51:50.715608 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=service%20pool%20kudu.tserver.TabletServerService
I20251027 02:51:50.715687 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=tablet-copy
I20251027 02:51:50.715761 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/threadz?group=thread%20pool
I20251027 02:51:50.715835 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/transactions
I20251027 02:51:50.715919 1729439 webserver-crawl-itest.cc:328] https://127.152.231.195:34915/varz
I20251027 02:51:50.715996 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701
I20251027 02:51:50.716069 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/bootstrap/js/bootstrap-table.min.js
I20251027 02:51:50.716143 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/bootstrap/js/bootstrap.min.js
I20251027 02:51:50.716216 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/config
I20251027 02:51:50.716288 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/healthz
I20251027 02:51:50.716367 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/ipki-ca-cert
I20251027 02:51:50.716444 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/jquery-3.5.1.min.js
I20251027 02:51:50.716519 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/kudu.js
I20251027 02:51:50.716594 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/logs
I20251027 02:51:50.716670 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/masters
I20251027 02:51:50.716743 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/mem-trackers
I20251027 02:51:50.716817 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/memz
I20251027 02:51:50.716895 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/metrics
I20251027 02:51:50.716971 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/metrics_prometheus
I20251027 02:51:50.717042 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/rpcz
I20251027 02:51:50.717139 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/stacks
I20251027 02:51:50.717216 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/startup
I20251027 02:51:50.717293 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/table?id=43231b1737d0464cafbecad8c62ddcb4
I20251027 02:51:50.717370 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/tables
I20251027 02:51:50.717446 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/tables?&redirects=1
I20251027 02:51:50.717525 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/tablet-servers
I20251027 02:51:50.717602 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz
I20251027 02:51:50.717680 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz?group=acceptor%20pool
I20251027 02:51:50.717778 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz?group=all
I20251027 02:51:50.717861 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz?group=cache
I20251027 02:51:50.717945 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz?group=catalog%20manager
I20251027 02:51:50.718024 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz?group=cloud%20detector
I20251027 02:51:50.718101 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz?group=kernel-watchdog
I20251027 02:51:50.718178 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz?group=maintenance
I20251027 02:51:50.718252 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz?group=master
I20251027 02:51:50.718329 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz?group=ntp
I20251027 02:51:50.718403 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz?group=reactor
I20251027 02:51:50.718478 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz?group=server
I20251027 02:51:50.718554 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:51:50.718629 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz?group=service%20pool%20kudu.master.MasterService
I20251027 02:51:50.718705 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:51:50.718782 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:51:50.718860 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz?group=tablet-copy
I20251027 02:51:50.718945 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/threadz?group=thread%20pool
I20251027 02:51:50.719022 1729439 webserver-crawl-itest.cc:328] https://127.152.231.252:46701/varz
I20251027 02:51:50.719097 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307
I20251027 02:51:50.719172 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/bootstrap/js/bootstrap-table.min.js
I20251027 02:51:50.719247 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/bootstrap/js/bootstrap.min.js
I20251027 02:51:50.719321 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/config
I20251027 02:51:50.719395 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/healthz
I20251027 02:51:50.719472 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/ipki-ca-cert
I20251027 02:51:50.719550 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/jquery-3.5.1.min.js
I20251027 02:51:50.719640 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/kudu.js
I20251027 02:51:50.719712 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/logs
I20251027 02:51:50.719785 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/masters
I20251027 02:51:50.719856 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/mem-trackers
I20251027 02:51:50.719935 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/memz
I20251027 02:51:50.720007 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/metrics
I20251027 02:51:50.720077 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/metrics_prometheus
I20251027 02:51:50.720168 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/rpcz
I20251027 02:51:50.720240 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/stacks
I20251027 02:51:50.720315 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/startup
I20251027 02:51:50.720391 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/tables
I20251027 02:51:50.720479 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/tablet-servers
I20251027 02:51:50.720551 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz
I20251027 02:51:50.720626 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz?group=acceptor%20pool
I20251027 02:51:50.720700 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz?group=all
I20251027 02:51:50.720775 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz?group=cache
I20251027 02:51:50.720849 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz?group=catalog%20manager
I20251027 02:51:50.720948 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz?group=cloud%20detector
I20251027 02:51:50.721030 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz?group=kernel-watchdog
I20251027 02:51:50.721128 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz?group=maintenance
I20251027 02:51:50.721237 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz?group=master
I20251027 02:51:50.721314 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz?group=ntp
I20251027 02:51:50.721390 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz?group=reactor
I20251027 02:51:50.721467 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz?group=server
I20251027 02:51:50.721540 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:51:50.721614 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz?group=service%20pool%20kudu.master.MasterService
I20251027 02:51:50.721691 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:51:50.721768 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:51:50.721848 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz?group=tablet-copy
I20251027 02:51:50.721930 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/threadz?group=thread%20pool
I20251027 02:51:50.722007 1729439 webserver-crawl-itest.cc:328] https://127.152.231.253:33307/varz
I20251027 02:51:50.722081 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179
I20251027 02:51:50.722158 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/bootstrap/js/bootstrap-table.min.js
I20251027 02:51:50.722234 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/bootstrap/js/bootstrap.min.js
I20251027 02:51:50.722316 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/config
I20251027 02:51:50.722390 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/healthz
I20251027 02:51:50.722468 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/ipki-ca-cert
I20251027 02:51:50.722550 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/jquery-3.5.1.min.js
I20251027 02:51:50.722626 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/kudu.js
I20251027 02:51:50.722703 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/logs
I20251027 02:51:50.722777 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/masters
I20251027 02:51:50.722851 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/mem-trackers
I20251027 02:51:50.722936 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/memz
I20251027 02:51:50.723012 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/metrics
I20251027 02:51:50.723088 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/metrics_prometheus
I20251027 02:51:50.723161 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/rpcz
I20251027 02:51:50.723234 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/stacks
I20251027 02:51:50.723308 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/startup
I20251027 02:51:50.723384 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/tables
I20251027 02:51:50.723460 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/tablet-servers
I20251027 02:51:50.723536 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz
I20251027 02:51:50.723692 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz?group=acceptor%20pool
I20251027 02:51:50.723819 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz?group=all
I20251027 02:51:50.723945 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz?group=cache
I20251027 02:51:50.724066 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz?group=catalog%20manager
I20251027 02:51:50.724186 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz?group=cloud%20detector
I20251027 02:51:50.724308 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz?group=kernel-watchdog
I20251027 02:51:50.724427 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz?group=maintenance
I20251027 02:51:50.724547 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz?group=master
I20251027 02:51:50.724666 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz?group=ntp
I20251027 02:51:50.724786 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz?group=reactor
I20251027 02:51:50.724913 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz?group=server
I20251027 02:51:50.725028 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz?group=service%20pool%20kudu.consensus.ConsensusService
I20251027 02:51:50.725172 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz?group=service%20pool%20kudu.master.MasterService
I20251027 02:51:50.725296 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz?group=service%20pool%20kudu.server.GenericService
I20251027 02:51:50.725419 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz?group=service%20pool%20kudu.tserver.TabletCopyService
I20251027 02:51:50.725543 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz?group=tablet-copy
I20251027 02:51:50.725661 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/threadz?group=thread%20pool
I20251027 02:51:50.725781 1729439 webserver-crawl-itest.cc:328] https://127.152.231.254:41179/varz
I20251027 02:51:50.729650 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1735667
I20251027 02:51:50.777644 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1735920
I20251027 02:51:50.843112 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1736215
I20251027 02:51:50.897073 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1735143
I20251027 02:51:50.942371 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1735298
I20251027 02:51:50.974303 1729439 external_mini_cluster.cc:1658] Killing /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/bin/kudu with pid 1735432
2025-10-27T02:51:51Z chronyd exiting
[ OK ] WebserverCrawlITest.TestAllWebPages/3 (96391 ms)
[----------] 4 tests from WebserverCrawlITest (267477 ms total)
[----------] Global test environment tear-down
[==========] 4 tests from 1 test suite ran. (267477 ms total)
[ PASSED ] 3 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] WebserverCrawlITest.TestAllWebPages/2, where GetParam() = (UseSsl::YES, ImpersonateKnox::NO)
1 FAILED TEST